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ABSTRACT 


The  convergence  of  solutions  to  the  discrete  or  sampled  time  linear  quadratic  regulator 
problem  and  associated  Riccati  equation  for  infinite  dimensional  systems  to  the  solutions 
to  the  corresponding  continuous  time  problem  and  equation,  as  the  length  of  the  sampling 
interval  (the  sampling  rate)  tends  toward  zero  (infinity)  is  established.  Both  the  finite  and 
infinite  time  horizon  problems  are  studied.  In  the  finite  time  horizon  case,  strong  continuity 
of  the  operators  which  define  the  control  system  and  performance  index  together  with  a 
stability  and  consistency  condition  on  the  sampling  scheme  are  required.  For  the  infinite 
time  horizon  problem,  in  addition,  the  sampled  systems  must  be  stabilizable  and  detectable, 
uniformly  with  respect  to  the  sampling  rate.  Classes  of  systems  for  which  this  condition  can 
be  verified  are  discussed.  Results  of  numerical  studies  involving  the  control  of  a  heat / diffusion 
equation,  a  hereditary  of  delay  system,  and  a  flexible  beam  are  presented  and  discussed. 
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1.  Introduction.  In  this  paper  we  consider  the  convergence  of  closed-loop  solu¬ 
tions  to  discrete  or  sampled  time  linear  quadratic  (LQ)  optimal  control  problems  and 
the  associated  Riccati  equations  for  infinite  dimensional  systems  defined  on  Hilbert 
spaces  to  the  solutions  to  the  corresponding  continuous  time  problems  and  Riccati 
equations,  as  the  length  of  the  sampling  interval  tends  toward  zero.  With  the  advent 
and  proliferation  of  micro-computers,  and  control  tasks  becoming  ever  more  complex 
(for  example,  the  stabilization  of  large  flexible  spacecraft),  the  roles  played  by  dis¬ 
crete  or  sampled  time  control  design  techniques  and  distributed  parameter  systems 
have  become  increasingly  more  important.  It  has  become  necessary,  therefore,  to 
develop  extensions  of  many  of  the  familiar  results  for  finite  dimensional  systems  to 
an  infinite  dimensional  setting.  One  area  that  has  recently  received  a  great  deal  of 
attention  has  been  the  LQ  theory.  Certain  aspects  of  the  linear-quadratic  approach 
to  control  design  for  both  continuous  and  sampled  time  infinite  dimensional  systems 
have  been  studied  extensively.  In  particular,  these  aspects  include,  for  example,  the 
linear  state  feedback  structure  of  the  optimal  control  law,  the  optimal  LQG  estimator 
and  compensator  problems,  boundary  control,  and  finite  dimensional  approximation 
(for  specific  references,  see  below).  But  to  the  best  of  our  knowledge,  however,  the 
inter-relation  between  the  continuous  and  discrete  time  theories,  which  in  the  finite 
dimensional  case  is  well  understood,  has  not  as  of  yet,  been  looked  at  in  the  context 
of  infinite  dimensional  systems.  Such  a  study  would  be  useful,  for  example,  because 
typically  in  engineering  practice,  the  discrete  and  continuous  time  LQ  theories  are 
applied  interchangeably  without  regard  to  as  to  whether  or  not  the  actual  system  is 
continuous  or  discrete  in  nature.  In  particular,  due  to  hardware  constraints,  most  sys¬ 
tems  occurring  in  engineering  practice  are  in  fact  discrete.  However,  if  the  sampling 
is  considered  to  be  rapid  enough,  the  system  may  be  treated  as  continuous  when  an 
optimal  control  law,  state  estimator,  or  compensator  is  designed.  Our  work  is  largely 
motivated  by  the  fact  that  the  results  we  shall  present  here  will  serve  to,  in  some 
sense,  justify  this  approach. 

We  note  that  in  finite  dimensions,  where  strong  and  uniform  norm  convergence  of 
linear  operators  are  equivalent,  the  continuous  dependence  with  respect  to  sampling  of 
the  solution  to  the  linear  quadratic  control  problem  and  associated  Riccati  equation 
is  straight  forward.  Indeed,  in  [Lej  the  continuous  time  theory  is  established  by 
first  deriving  the  discrete  time  results,  which  are  fundamentally  algebraic  in  nature, 
and  then  taking  the  limit  as  the  length  of  the  sampling  interval  tends  toward  zero. 
However,  in  infinite  dimensions,  as  is  typically  the  case,  the  desired  convergence  is 
less  obvious.  This  is  especially  true  in  the  case  of  the  infinite  time  horizon  problem. 
It  is  this  problem  that  we  address  here. 

We  consider  both  the  finite  and  infinite  time  horizon  problems.  In  the  case  of 
the  finite  time  horizon  problem,  under  the  assumption  of  strong  continuity  of  the 
operators  which  define  the  control  system  and  performance  index,  together  with  a 
stability  and  consistency  hypothesis  on  the  sampling  scheme,  we  are  able  to  deduce 
the  desired  convergence.  We  must  develop  an  appropriate  framework  to  facilitate  the 
comparison  of  discrete  and  continuous  time  operator  families.  For  this  purpose  we  rely 
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heavily  upon  Kato’s  [K]  treatment  of  discrete  semigroups.  In  the  case  of  the  infinite 
time  horizon  problem  we  must  additionally  assume  stabilizability  and  detectability  of 
the  discrete  time  systems  with  some  degree  of  uniformity  in  the  sampling  rate.  The 
notion  of  stabilizability /detectability  uniform  with  respect  to  sampling  will  be  made 
precise  in  Section  3  below.  We  are  able  to  establish  that  if  the  continuous  time  system 
is  stabilizable  and  detectable  via  finite  rank  feedback,  and  if  zero-order  hold  sampling 
is  employed,  then  the  resulting  discrete  time  systems  are  uniformly  stabilizable  and 
detectable  for  sufficiently  small  sampling  interval.  We  also  have  a  result  concerning 
the  uniform  stabilizability  and  detectability  of  parabolic  systems.  However,  this  result 
will  not  be  discussed  here,  but  rather  in  a  forthcoming  manuscript. 

Our  treatment  is  functional  analytic  in  nature,  and  is  similar  in  spirit  to  the  many 
recent  studies  of  convergence  of  solutions  to  LQ  control  and  estimation  problems 
and  the  associated  Riccati  equations  under  state  (space)  approximation  (i.e.  finite 
difference,  modal,  or  finite  element,  for  example).  See,  for  example,  [BK],  [BW], 
[G],[GA],  [GR],  and  [Wj.  For  the  discrete  time  LQ  theory  for  infinite  dimensional 
systems,  we  rely  heavily  on  the  well  known  results  contained  in  [HH],  [LCB],  and  [Z]. 

In  addition  to  our  theoretical  results,  we  have  included  the  results  of  some  of  our 
numerical  convergence  studies.  We  present  and  discuss  our  findings  for  the  infinite 
time  horizon  LQ  optimal  control  problems  for  a  one  dimensional  heat  or  diffusion 
equation,  a  one  dimensional  hereditary  or  delay  system,  and  a  hybrid  system  of  or¬ 
dinary  and  partial  differential  equations  describing  the  small  amplitude  transverse 
vibration  of  a  cantilevered  Voigt-Kelvin  viscoelastic  beam  with  tip  mass. 

An  outline  of  the  remainder  of  the  paper  is  as  follows.  In  section  2  we  treat  the 
finite  time  horizon  problem.  The  infinite  time  horizon  problem  is  considered  in  the 
third  section.  Our  numerical  results  are  presented  and  discussed  in  Section  4,  while 
a  brief  fifth  section  contains  a  summary  and  some  cor  ,  j^mg  remarks. 

2.  LQR  Problems  with  Finite  Time  Horizon.  .  this  section  we  consider 
the  linear  quadratic  regulator  (LQR)  problem  over  a  finite  time  interval.  The  basic 
notation  and  our  general  assumptions  are  introduced  in  the  statements  of  both  the 
continuous  time  and  corresponding  sampled  time  problems  given  below.  The  existence 
and  uniqueness  of  the  optimal  control  a s  well  as  its  closed  loop  feedback  structure  can 
be  obtained  using  a  variety  of  approaches.  Here  we  opt  to  consider  the  optimal 
control  problem  as  the  minimization  of  a  strictly  coercive  quadratic  form  on  the 
admissible  control  space.  _  This  approach  yields  an  explicit  representation  for  the 
solution  of  the  usual  Riccati  equations  (for  both  the  continuous  and  sampled  time 
problems)  in  terms  of  the  underlying  system  and  penalty  operators  which  define  the 
problems.  Since  the  particular  focus  of  our  effort  here  is  the  consideration  of  sampled 
time  problems  as  approximations  to  a  continuous  time  problem,  specialized  notions 
and  characterizations  of  convergence  must  be  introduced.  Once  this  is  done,  our 
fundamental  result  for  the  finite  time  horizon  problem  can  be  stated  in  terms  of  these 
specialized  notions  of  convergence  as  follows.  The  convergence  of  the  optimal  control 
and  the  optimal  feedback  laws  for  the  sampled  systems  to  the  optimal  control  and 
feedback  law  for  the  continuous  time  problem  as  the  length  of  the  sampling  interval 
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tends  to  zero,  follows  directly  from  the  convergence  of  the  open-loop  sampling  of  the 
underlying  linear  control  system  and  quadratic  performance  index.  We  consider  the 
open-loop  sampling  of  the  infinite  dimensional  LQR  problem  in  an  abstract  setting  so 
that  our  results  can  be  applied  to  a  wide  range  of  sampling  procedures. 

Let  H  and  U  be  Hilbert  spaces  with  inner  products  <  •  >h  and  <  •  >u 
respectively.  Let  t0,t /  G  R  be  given  with  to  <  t/,  and  let  T  =  {T(t,s)  :  t0  <  s  < 
t  <  tf}  be  an  evolution  system  on  H.  For  each  f  G  [to,*/],  let  B(t)  G  L(U,H), 
Q(t)  6  L(H),  and  R(t)  G  L(Z7),  and  let  G  G  L(H).  We  consider  the  continuous  time 
LQR  problem  given  by 

(P)  Determine  a  control  input  u  G  Li{to,tf\U)  which  minimizes  the  quadratic  per¬ 
formance  index 

J{u\t0,x{t0),G)  =  <  Gx(tf),z(tf)  >H 

+  [  '  {<  Q(t)x(t),x(t)  >H  +  <  R(t)u(t),u(t)  >u}dt 
Jto 

where  for  each  t  G  [to,tf]  the  state  x(t)  G  H  is  given  by 

(2.1)  x(t)  =  T(t,s)x(s )  +  J'  T(t,T)B(r)u(T)dr ,  t0  <  s  <  t  <  tf. 

We  make  the  following  standard  assumptions  on  the  operator  families  {T,  B ,  G,  Q,  R} 
which  determine  problem  (P). 

(Cl)  The  evolution  system  T  is  strongly  continuous  on  H  and  therefore  is  uniformly 
exponentially  bounded,  with  constants  M  >  0  and  u>  G  R.  That  is 

lirMIUw < t0<s<t<tf. 

(C2)  The  operator  valued  functions  B,Q,  and  R  are  strongly  continuous  and  there¬ 
fore  are  uniformly  bounded  on  }to>t/]>  That  is,  there  exists  a  constant  C  >  0 
for  which 

\\B(t)\\L(UiH)  <  C,  ||Q(0IU(/r)  <  C,  ||-R(0lk(^)  ^ 

t  G  [t0,  t/\. 

(C3)  The  operator  G  and  the  operators  Q(t)  and  R(t)  for  each  t  G  \to,tf  \  are  self- 
adjoint  and  nonnegative  definite.  Moreover,  there  exists  a  constant  r  >  0  for 
which  R{t)  >  rl,t  G  [to,*/]- 

The  strong  continuity  assumption  in  (C2)  is  not  necessary  for  the  well-posedness 
of  the  LQR  problem.  However,  some  assumptions  on  the  continuity  of  the  operators 
B,  Q,  R  will  be  needed  to  obtain  uniform  convergence  with  respect  to  sampling. 

The  closed-loop  linear  state  feedback  form  of  the  solution  to  problem  (P)  can 
be  shown  to  exist  and  be  explicitly  construe  led  by  considering  the  minimization  of 
appropriately  constructed  strictly  coercive  quadratic  forms  on  the  Hilbert  spaces  U,  — 

L/2 (s, tj\  U) ,  s  G  [t0  tf]  (see,  for  example,  [G]).  Since  it  will  play  a  prominent  role  in 
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our  discussions  to  follow,  we  briefly  outline  this  approach  here.  For  each  s  6  [to,*/] 
define  the  operators  8,  G  L[H,  U,)  and  Z,  €  L(Ua)  by 

(2.2)  (8,4>){t)  =  B(t)’  l[T{tf,tyGT{tf,s)+jttfT{r1,tyQ{r1)T{V,s)dri)j4>, 

for  <f>  €  H,  t  6  and 

(2.3)  (je,u,)(t)  =  R(t)u,(t)+B(tYT{t„tYG  y  T(thv)B(ri)u.Wdri 

+B(tY  I''  {r(u,  «)•«(!,)  I"  T(i7,r)B(r)«,(r)ir}  dr,, 

for  t  €  [ s,tf ]  and  u3  €  U„.  It  is  not  difficult  to  verify  that  the  adjoint  operator 
8;  G  L(U,,H)  of  8,  is  given  by 

(2.4)  B,u,  =  T{tf,SyG  I,' T{tf,t)B(t)u,{t)dt 

+  j  f  T{t,s)’Q{t)  {f  T{T,ri)B(r))u,(ri)dTi^  dr , 

and  that  for  u,  6  £/a,  we  hive 

J(u,;s,x(s),G)  =<  GT(t/,s)x(s),r(t/,s)x(s)  >H 

+  J  f  <  <2(t)T(f,s)x(s),T(t,s)x(s)  >H  dt-  <  Z~18,x(s),8,x(s)  >u. 

+  <  £,(u,  +  ^718ax(s)),uJ  +  Z~l8,x{s)  >Ut  . 

It  follows  that  for  x(s)  €  #  given,  J(-;  s,  x(s),  G)  is  minimized  by  choosing  u,  =  u,  = 
—  Z~lB,x(s)  €  il,.  We  then  obtain 

min  J(-;s,x(s),G)  =  J(uf;s,x(s),G) 

=  <  GT(tf,s)x(s),T(tf,s)x(s)  >h  + 1*'  <  Q(t)T(t,s)x(s),T(t,s)x(s )  >H  dt 

-  <  Z~lB,x(s),  8,x(s)  >u. 

=  <  II(s)x(s),x(s)  >H 

where  the  self-adjoint  operator  valued  function  IT  :  [*o,t/|  <— ►  L(H )  is  defined  by 

(2.5)  n(s)d>  =  T(tf,s)*GT(tf,s)</>  +  J*'  T(t,syQ(t)T(t,s)<j>dt 

-8;z:x8a<t>,  <t>e  H. 

Using  the  definitions  given  above,  the  following  theorem  concerning  the  existence  and 
characterization  of  the  closed-loop  solution  to  problem  (P)  can  be  established. 

THEOREM  2.1.  Suppose  that  assumptions  (Cl)-(C3)  are  satisfied.  Then  for 
any  initial  state  x(*o)  -  H  given,  there  exists  a  unique  solution  u  to  problem  (P).  The 
optimal  control  u  is  given  in  linear  state  feedback  form  by 

u(t)  —  —  i2(t)_1£(t)*II(f)x(t), t  €  [to,*/] 
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where  x  is  the  optimal  trajectory.  The  operator  valued  function  II  is  given  by  (2.5) 
and  it  is  the  unique  self-adjoint  solution  to  the  Riccati  integral  equation 


(2.6)  n(t)  =  T(tf,t)'GT{tf,t) 

+  f’’  T{r,tY  {Q(t)  -n(r)B(r)R(T)-'B(ryn(r)}T(r,t)dr, 

t  £  [t0,tf\.  We  have 

(2.7)  min  J  (•;  t0,  x(to),G)  =  J(u-,t0,x{t0),G )  =  <  n(f0)x(t0).x(*o)  >h  • 

u  • 


We  consider  next  the  discrete  or  sampled  time  problem.  Let  k0,kf  £  Z  with 
kf  >  k0  and  let  h  £  R  with  h  >  0.  For  k  €:  Z  with  ko  <  k  <  kf  —  1  let  Ah{k)  £  L(H), 
and  let  {Th(k,j)  :  k0  <  j  <  k  <  kf}  be  the  discrete  time  evolution  system  on  H  given 
by 


k-l 

Th{k,k)  =  /,  Th{k,j)  =  Ah{k-l)-Ah{k-2)  ■  ■  ■  Ah(j)  =  II  M'h  k0  <  j  <  k  <  kf. 
(2.8) 

Let  {Bh(k)}kkLk1o,  {Qh{k)}kkf=kl,  and  {Rh(k)}kLko  be  sequences  in  L{U,H),  L{H)  and 
L(U)  respectively,  and  let  Gh  £  L(H).  The  LQR  problem  is  then  given  by 

(P*)  Determine  a  control  input  uh  £  l2[ko,kf  —  1;  U)  which  minimizes  the  quadratic 
performance  index 

Jh {uh’i  ko,  Xh (A^o) ,  Gh)  =<  GhXh{kf),Xh{kf) 

+h  {<  Qh{k)xh(k),xh{k)  >H  +  <  Rh[k)uh(k),uh(k)  >[/} 

k=k0 

where  for  each  k  £  Z  with  k0  <  k  <  kf,  the  state  Xh(k)  £  H  is  given  by 

k- 1 

(2.9)  xh{k)  =  Th{k,j)xh(j)  +  hY,Th(k,i  +  !)£*(*>*  (t), 


for  k0  <  j  <  k  <  kf. 

For  the  discrete  time  case,  we  make  the  following  assumptions. 

(Dl)  For  each  h  >  0  the  operators  An(k),  Bh{k),  Qh{k),  and  Rh(k )  are  bounded  in 
k  for  k0  <  k  <  kf  —  1.  Thus,  there  exists  a  constant  C ^  for  which 

II  A^(/c)  ||r,(«-)  <  Ch,  \\Bh(k)\\L(u,H)  <  Ch, 
iiQh(^)||i(H)  <  Ch,  II^WIUw  ^  Chi 


for  k0  <  k  <  kf  —  1. 
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(D2)  The  operator  Gh  and  the  operators  Qh(k)  and  Rh(k)  for  Ar0  <  k  <  kf  —  1  are 
self-adjoint  and  nonnegative.  Moreover,  there  exists  a  constant  >  0  for 
which  Rh{k)  >  rhI,  k0  <  k  <  kf  —  1. 

Note  that  assumption  (Dl)  together  with  (2.8)  yield  that  the  discrete  time  evolution 
system  { Th{k,j )  :  fc0  <  j  <  k  <  kf}  is  uniformly  exponentially  bounded  with 

\\Th{k,j)\\L(H)  <  Cj~\  kQ<j<k<  kf. 

Note  also  that  the  discrete' time  evolution  equation  (2.9)  is  equivalent  to  the  discrete 
time  dynamical  system  given  by 

(2.10)  Xh(k  +  1)  =  Ah(k)xh{k)  +  hBh(k)uh(k),  k0  <  k  <  kf  -  1, xh(k0)  £  H. 

For  each  h  >  0  and  j  =  k0,  k0  +  1,  •  •  • ,  kf  —  1,  let  Uk  j  =  h (j,  kf  —  1;  U)  endowed 
with  the  inner  product 


<  uh,j > Vh,j  —  k  53  <  Uhfj{^k),Vh,j{k)  >u  . 

*=; 

Define  the  operators  8hJ  £  L(H,Uh,j)  and  Zhij  £  L(Uhij)  by 

(2.11)  (Bhtj4>)(k)  =  Bh(kYTh(kf,k+  lYGhTh(kf,j)<t> 

{  */-i 

(  «=*+l 

for  <f)  £  H,  k  =  j,  j  +  1,  •  •  • ,  kf  -  1,  and 

(2.12)  (£fclittW)(fc)  =  Rh(k)uhj(k) 

*/-» 

+  Bh(k)*Th{kf,k  +  l)*Ghh  £  Th[kf,i  +  l)Bh(i)uhj(i) 


•=) 


i—  1 


+5,(A:)*h  53  rfc(*\*  +  i)*gfc(0  Oi£r„(M  +  i)b,(/k ,(/)  , 


t=jfc-t- 1 


i=j 


£  ^*,y,  k  —  j,j  +  1,  •  •  • , kf  —  1,  respectively,  where  in  the  above  expressions  and 
throughout  the  remainder  of  the  paper  we  adopt  the  convention  that  M  a,  =  0 
whenever  v  <  n.  It  is  not  difficult  to  verify  that  the  adjoint  of  B^j,  the  operator 
B'h j  £  L(Uh  j,  H)  is  given  by 

kf~i 

(2.13)  BZjUbj  =  Tk{kfjyGKh'ETh{kf,k+l)Bh{k)ukJ{k) 

*/_1  (  fc-i  ) 

h  E  Mk,j)-Qh{k)  fcj;r,(*;,i+l)Bk(<)ukJ(.)  , 

k=j+l  {  i=i  I 
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for  uhJ  G  Uhj. 

Proceeding  as  we  did  in  the  continuous  time  case,  we  find  that  for  j  =  ko,  ■  ■  ■ ,  kf  — 
1,  xh{j)  €  H,  and  vhJ  G  U'h>] 

Jh{uh,j-J,xh{j),Gh)  =  <  GhTh{kf,j)xh{j),Th(kf,j)xh(j)  >H 

kf-i 

+hY,<  Qh{k)Th(k,j)xh{j),Th(k,j)xh(j )  >H 
k=j 

-  <  ^hjBhijxh{j),Bh,jXh{j)  >ukJ 

+  <  %h,j{uh,j  d"  ^’h,jBh,jxh{j))^uh,j  +  >Uh.p 

where  the  existence  of  the  inverse  of  R^.j  is  quaranteed  by  assumption  (D2).  It 
is  immediate'y  clear  that  for  j  G  Z  with  j  G  [k0,kf  —  l]  and  x/,(j)  6  H  given, 
Jh{-',j,Xh{j),Gh)  is  minimized  when  uhtj  =  uhj  =  -Zhj8h,jXh(j).  It  follows  that 

min  Jh{-J,xh{j),Gh)  =  Jh{uhJ\j,xh(j),Gh) 

Uh,j 

=  <  GflTh(kfJ)xh(j),Th{kf,j)xh(j)  >h 
kf~ 1 

+h  £  <  Qh(k)Th(k,j)xh(j),Th(k,j)xh(j)  >H 

k=j 

~  <  %-h,j  BhjXh(j)  >Uk,j 

=  <  nh{j)xh{j),xh{j)  >H, 

k  i 

where  the  sequence  of  self-adjoint  operators  in  L(H),  {11^  (/:)}*' fco,  are  given  by 

kf-l 

(2.14)  n h{j)<f>  =  Th(kfJYGhTh(kf,j)<t>  +  h  £  Th(k,j)'Q,Xk)Tk{kJ)<l> 

k= ; 

—  8hjRkj8h,j<f>, 

for  k  =  A:0,  •  •  • ,  kj  —  1  and  <f>  G  H.  We  note  that  it  is  completely  consistent  to  define 
Hk(fc/)  =  Gh- 

Using  the  above  definitions,  it  is  possible  to  establish  the  following  well  known 
result  (see,  for  example,  [LCB],  [Z],  and  [GR])  for  the  discrete  time  LQR  problem 

(P0- 

THEOREM  2.2.  Suppose  that  assumptions  (Dl)  and  (D2)  are  satisfied.  Then  for 
any  given  initial  state  Xh(k0)  G  H  there  exists  a  unique  solution  u ^  G  ^(^o^/  —  1;  U) 
to  problem  (Ph)-  It  is  given  in  linear  state  feedback  form  by 

uh(k)  =  -Rk(k)~1Bh{kynh{k  +  l)Ah(k)xh(k),  k  =  k0,  •  •  • ,  kf  -  1, 

where  Rh{k)  =  Rh{k)  +  hBfl{k)*Tlfi(k  +  l)Bh(k),  for  k  =  k0,  •  ■  • ,  kf  —  1,  and  the  optimal 
trajectory  z/,  is  given  by  (2.9)  ( equivalently  (2.10))  with  u ^  =  uh.  The  sequence  of 
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operators  in  L(H),  {IMA;)}£Lfc*  are  given  by  (2.14)  with  IMA:/)  =  Gh  and  can  be 
obtained  recursively  via  the  Riccati  difference  equation 

(2.15)  IMA:)  =  Ah{kynh{k  +  l)A,{k) 

-hAH{kynh[k  +  l)B,(At)^(A:)-15,(A:)*n,(A:  +  1  )Ah{k) 
+hQh(k), 

k  =  kf  —  1, •  •  • , k0)  n^(A:/)  =  Gh-  We  have 

(2.16)  min  Jh{",  k0,xh(k0),Gh)  =  Jh{uh-,  k0,  xh(k0),  Gh) 

“A.*0 

=  <  nh{ko)xh(k0),zh(ko)  >H  . 

For  appropriate  choices  of  the  families  of  operators  7\, Bh,Qh,  and  Rh,  we  are 
interested  in  studying  the  convergence  of  solutions  to  the  problems  (P/,)  to  the  solution 
of  problem  (P)  as  the  length  of  the  sampling  interval,  h ,  tends  toward  zero.  In 
particular,  we  want  to  investigate  the  convergence  of  the  discrete  families  of  Riccati 
operators  {TT/v(/c)  :  ko  <  k  <  kf}  to  the  continuous  family  of  operators  {IT(t)  :  to  < 
t  <  t}). 

In  '  'der  to  reduce  the  necessary  degree  of  technical  detail,  we  make  the  simplifying 
assumption  that  t0  =  0.  There  is  of  course  no  loss  of  generality  in  doing  this  since 
any  system  can  be  transformed  to  one  on  a  time  interval  starting  at  the  origin.  Set 
Aro  =  0  and  for  each  h  >  0  let  kf  =  kfh  =  [tf/h]  where  for  a  6  R,  [a]  is  used  to 
denote  the  greatest  integer  less  than  or  equal  to  a.  Let  tf:h  =  hkf  h  and  note  that 
lim/l_0+  tfih  =  tf. 

In  order  to  compare  discrete  and  continuous  families  of  operators,  it  is  useful  to 
identify  certain  /2  sequence  spaces  with  subspaces  of  L2.  For  X  a  Hilbert  space  and 
all  h  >  0,  let  L2,h{0,tftfl;  X)  be  the  subspace  of  Li(0,tfy,X)  defined  by 
-^2,^(0,  tfy,  X)  =  {(f>  €  L2(0,  tfX,  X)  :  <j)  is  constant  on  each  of  the  intervals 
[0,  h),  [h,  2/i),  •  ■  • ,  [{kf'h.  -  1  )M/.*)}- 

Note  that  the  subspace  L2|>,(0,  tf]h]  X)  of  L2(0,  tfy.  AT)  is  isometrically  isomorphic  to 
the  space  I2(0 ,  A;/^  —  1;  AT)  endowed  with  the  inner  product 

<  >=  h  YL 

j=o 

Let  U  =  L2(0,  tf\U)  and  let  Uh  =  L2,h{0,  tfth\ U).  Let  Ph  €  L{U,Uh)  be  the 
orthogonal  projection-like  mapping  of  U  onto  Uh  defined  by 

—  i 

(Ph4>){t)  =  Y2  MjXijit),  o<t<tfih, 

}=0 

for  <f>  €  U  where  for  j  —  0, 1,  •  •  • ,  kf<h  —  1,  X/;  is  the  characteristic  function  for  the 
interval  =  [jh,  ( j  +  1  )h)  and 

=  h~l  f  <j>(t)dt. 
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It  is  not  difficult  to  show  (see  Appendix  A)  that 

(i)  the  net  {|)-PA }^>o  is  uniformly  bounded; 

(ii)  limh-.0+  \\Ph<j>\\uK  =  11^1! a,  <t>  £  ii,  and 

(iii)  for  each  0  £  U/,  there  exists  a  0  £  U  such  that  0  =  PA0  and  j|0||^  =  ||0||ufc. 
Following  Kato  [K,  §IX.4]  we  say  that  a  net  {<t>h}h>oi<t>h  €  Uh  converges  to  0  6  ii 

{4>H  -*  0,  or  lim/,_0+  4>h  -  <t>)  if 

lim  |!0a  -  Pk0||Uk  -  o. 

n — ►O' 

Also,  if  for  h>  0,  0h  £  L(Uh),  then  we  say  that  0*  converges  strongly  to  0  €  L(U)  if 
$hPh<t>  —*  00,  0  €  ii;  that  is  if 

lim  ||0*Pa0  -  P>&4>\\uk  =0,  6  ell. 

h — >0+ 

With  strong  operator  convergence  defined  in  this  way,  it  can  be  shown  that  0/,P*0  — ► 
00,  0  e  ii  implies  that  the  net  {||0h||L(ufc)}  is  uniformly  bounded  and  that  if  0>,Ph0  — ♦ 
00, and  0^P/,0  — 1 >  00,  0  €  ii,  then  0h0kn>»0  — *■  000,  0  £  ii,  etc.  We  note  of  course 
that  an  analogous  definition  of  strong  convergence  can  be  made  for  bounded  operators 
having  only  one  or  the  other  of  its  domain  and  co-domain  being  ii*.  That  is,  for 
example,  if  0/,  €  L(X,U h)  and  0  £  L(X,U)  where  X  is  a  normed  linear  space,  then 
we  say  that  0,,  converges  strongly  to  0  if  0*x  — *•  0x,  x  £  X,  or 

lim  ||0fcx  -  P*0x||zu  =  0. 

a — ►0+ 

Following  the  treatment  of  discrete  semigroups  in  Kato  [Kj,  we  make  the  following 
formal  definition. 

DEFINITION  2.1.  The  discrete  time  families  of  bounded  linear  operators  0>,  = 
{0/,(fcn,  fcn- 1,  •  •  • ,  ki)  :  0  <  k\  <  fc2  <  •  •  •  <  kn  <  Kh},  h  >  0  from  a  Banach  space 
X  into  a  Banach  space  Y  will  be  said  to  (strongly)  approximate  a  continuous  time 
family  of  operators  0  =  {0(tn,  tn_j,  •  •  • ,  tj)  :  0  <  <  t2  <  •••  <  tn  <  T}  with 
0(<n,---,ti)  €  L(X,Y)  for  t  =  tx)  £  A  (n,T)  =  {(tn,tn-u  •  •  • ,  £i)  £  Pn  :  0  < 

h  <  h  <•••<  tn  <  T},  at  t  =  (£n, £  A(n,T),  if 

(i)  There  exists  at  least  one  net  of  multi-indices  {kh  }/»o,  k  -  (kn  h,  ■  ■  ■ ,  kiih)  £  Zn 

t  A  A  ^  A  A 

with  0  <  ki'h  <  •  •  •  <  kn<h  <  Kh  and  lim^_o+  hkh  =  t. 

(ii)  For  all  nets  {kh}h> o,  satisfying  (i)  above, 

lim  ||0/,(fc/,)i  -  0(£)i||k  =  0,  x  £  X. 

The  families  0h,/i  >  0  will  be  said  to  approximate  0  on  the  set  A(n,P),  if  Kh  —  [T/h] 
and  if  0*  approximates  0  at  each  t  £  A(n,T'). 

When  the  discrete  time  families  0/,,  h  >  0  approximate  the  continuous  time  family 
0  at  time  t  (on  the  set  A(n,T))  we  shall  write  0*  — ♦  0  at  time  t  (on  the  set  A(n,T)). 

Definition  2.2.  For  h  >  0  and  $h  =  {$h(kn,kn-U  ■  •  •  ,ki)  :  0  <  kt  <  k2  < 
•••  <  kn  <  Kh}  a  discrete  time  family  of  bounded  linear  operators,  we  define  an 
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associated  continuous  time  family  of  operators,  =  {$/,(£n»  tn-u  ■  •  •  ,  ti)  •'  0  <  tx  < 
t2  +  h  <  ■■■  <  tn  +  h  <  (Kh  +  l)/i}  via  Sh(t„,  ■  ■  ■ ,  tx)  =  $h{[tn/h],  ■■■,  fa/fe])  for 
t  =  (*n, •••,*!)  e  A h{n,Kh)  =  {(*„,*„_ ti)  6  Rn  :  0  <  tx  <  t2  +  h  <  ■■■  < 
tn  +  h  <  (Kh  +  l)h}. 

Note  that  when  Kh  =  [T/h],  A (n,T)  C  A h[n,Kh)  for  all  h  >  0. 

The  proof  of  the  following  theorem  can  be  argued  in  much  the  same  manner  as 
were  the  proofs  of  Lemmas  IX. 3. 4  and  IX. 3. 5  in  Kato  [K]. 

THEOREM  2.3.  Suppose  that  the  continuous  time  family  of  bounded  linear  opera¬ 
tors  $  is  strongly  continuous  on  A  (n,T)  and  that  #/,,  h  >  0  are  discrete  time  families 
for  which  $h  —*  $  on  the  set  A (n,T).  Suppose  further  that  for  each  h  >  0,  is 
the  continuous  time  family  on  A h(n,  Kh)  corresponding  to  the  discrete  time  family  $h 
constructed  according  to  Definition  2.2  above.  Then 

(i)  The  families  h>  0  are  uniformly  bounded  in  h  in  L(X,Y );  that  ts  there 
exists  a  constant  M  >  0  independent  of  h  for  which 

||$h(&r»>  kn-i,  •  •  • ,  fcOlUpr.y)  <  M,  0  <  kx  <  k2  <  •  •  •  <  kn  <  Kh,h  >  0, 

(ii)  $h  ~ +  $  uniformly,  in  t  for  t  £  A (n,T);  that  is 

Um  ||$Jt)x  _  $(Oz!ly  =  o,z  e  x, 

h—0+ 

uniformly  in  t  for  t  =  (tn,  •  •  •  ,ti)  €  A(n,T). 

Conversely,  if  Kh  —  [T/h]  and  —*  $  uniformly  in  t  for  t  £  A (n,T),  then  — ►  $ 
on  the  set  A (n,  T) . 

Let  the  continuous  time  families  T  =  :  0  <  s  <  t  <  T}  C  L(H), 

B  =  {B[t)  :  0  <  t  <  tf}  C  L(U,H),  Q  =  { Q(t )  :  0  <  t  <  tf)  C  L(H)  and 
R  =  {R(t):  0  <  t  <  tf}  C  L(U)  be  as  given  in  the  statement  of  the  continuous  time 
LQR  problem  (P)  (i.e.,  in  particular  assume  that  the  conditions  (Cl)-(C3)  hold). 
For  h  >  0,  let  kf<h  =  \tffh]  and  ’°t  Ah  =  {A h(k)  :  0  <  k  <  kfih  -  1}  C  L(H), 
Bh  =  [Bh(k)  :  0  <k  <  kfih  -  1}  C  L(U,  H ),  Qh  =  {Qh(k)  :  0  <  k  <  kf<h  -  1}  C  L(H), 
and  Rh  =  {Rh(k)  :  0  <  k  <  kj  h  —  1}  C  L(U)  be  discrete  time  families  of  bounded 
linear  operators  which  satisfy  conditions  (Dl)  and  (D2)  and  which  satisfy  the  following 
conditions. 

(Al)  Bh  —>  B,  Qh  —*■  Q,  Rh  — 1 ■  R,  and  B*h  -»  B*  on  the  set  A(l,£/)  where  B *  = 
{B(*Y  :  0  <  t  <  tf}  and  B’h  =  {Bh(ky  :  0  <  k  <  *,,*}. 

(A2)  (a)  (Stability)  The  discrete  time  families  of  operators  Th  =  {Th(k,j)  :  0  <  j  < 
k  <  kf'h}  C  L(H )  given  by 

rfc(*.y)  -  {  § 1<k' 

{  D  j  =  k 

are  uniformly  bounded  in  L(H )  for  h>  0. 
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(b)  (Consistency) 

lim  ^\\fh(t  +  h ,  t)(j>  -  T{t  h,  t)<j>\\  =0,  <j>  €  H, 

h— 0+  h 

and 

lim  h fH{t  +  -  T(t  +  h,ty<f>\\  =  o,  <t>  e  h, 

h-.o+  h 

uniformly  in  t  for  t  G  [0,f/]. 

(A3)  The  scalars  given  in  the  statement  of  condition  (D2)  are  bounded  away  from 
zero  uniformly  in  h.  That  is  rh  >  r  >  0,  h  >  0. 

LEMMA  2.1.  Condition  (AS)  implies  that  T),  — ►  T  and  T)*  —*  T*  on  the  set 

A(2 ,«/). 


Proof.  We  consider  the  convergence  — ►  T  only;  the  adjoint  convergence  is  com¬ 
pletely  analogous.  Following  the  proof  of  the  well  known  Lax-Equivalence  Theorem 
[RM],  the  result  is  an  immediate  consequence  of  condition  (A2),  the  strong  continuity 
of  the  continuous  time  family  T,  and  the  identity 


Th{k,j)<j>-T[kh,jh)<f> 


k- 1 


E  Th{k,i  +  l){Afc(0  -  T((i  +  1  )h,  ih)}T(ihJh)<j>, 


•-j 


(  0, 


k  >  j, 

k  =  j, 


0  <  j  <  k  <  kf<h,  <f>€  H.  □ 

We  shall  also  assume  that  G  €  L(H)  is  a s  in  condition  (C3)  and  that  for  each 
h  >  0  the  operator  Gh  €  L(H)  satisfies  condition  (D2).  We  require  that  the  additional 
approximation  condition 
(A4)  limh_0+  Gh<p  =  G(f>,  <p  €  H , 
be  satisfied  as  well. 

For  h  >  0  and  s  6  [0,  tj]  define  Bh,,  €  L{H,Uh)  by 


(2.17) 


(&h',<t>)(t) 


For  t  £  [0,  i -  h]  : 

X{[3/k\k,tIM\{t)Bh{ty {fh(tfth,t  +  h)‘ GhTh[tf_h,s) 
+  f  1  Th(ri,t  +  h)’Qh(ri)fh(ri,$)dri}<l>, 

<  J[{t+h)/h}h 


For  t  €  [tf'h  —  h,tf'h\  : 
,  (Bh,ad>){tf,h  -  h), 


when  s  6  [0,  tf^),  and  by  ( Bh =  0  when  s  €  {tf,h,tf\,  for  4>  €  H.  Note  that  for 
j  =  0, 1,2,-  •  •  ,kLh  -  1,  k  =  jj  +  1,  •  •  • ,  k f  h  -  1,  and  <f>  €  H 

(2-18)  (B„»(t)  =  (Bhj<f>)(k), 

for  s  €  [jh,(j  +  l)h)  and  t  €  [fch,  (/c  -I-  l)h),  where  for  j  =  0, 1, 2,  •  •  • ,  kf^  —  1, 
Bh <E  L(U,Uh,j)  is  given  by  (2.11). 
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Similarly,  for  5  £  [0,4/]  define  £  L(Uh,H)  by 


(2.19)  B*h,Uh  —  Th(tLh,s)'Gh  f  fh{tj'h,t  4-  h)Bh{t)uh(t)dt 

•/[.s/hjh 

/■*/,*  -  (  r\T/h]h  ~ 

+  r»(r,i)-g»(r)  /  T»(r,,  +  fc)B»(, 

I  j\slh\h 


Th(r,syQh(r) 


rlr/h]h  .  .  ) 

I  Th(r,r)  4-  h)Bh(v)uh(Tj)dr)  >  dr, 
[a/h]h  J 


when  s  €  (0,4/,*),  and  by  8^3uh  =  0  when  5  £  [4/^,4/j  for  uh  £  Note  that  for 
y  =  0, 1, 2,  •  •  • ,  A//,  -  1  and  uhj  £  Uhj  we  have 

(2-20)  B;.uk  =  BijUH 

for  s  £  [jh,  (j  +  l)h)  when  u*  £  Uh  is  given  by 


(2.21) 


uh(t)  = 


0,  0  <  t  <  jh 

Uhj(k),  kh  <  4  <  (A:  4-  l)h, 


k  =  j,j  4-  1  ,"-,kfth  —  1,  and  S^;  £  L(Uhj,H)  is  given  by  (2.13).  Note  also  that 
&h,a  =  (8h,s)*-  That  is  that  £  L(Uh,H)  given  by  (2.19)  is  the  Hilbert  space 
adjoint  of  the  operator  Bh,,  €  L(H,Uh)  given  by  (2.17)  for  all  s  £  [0,4/]. 

For  s  £  [0,4/]  define  Rh,a  £  L(Uh)  by 

For  4  £  [0, 4 fih  -  h\  : 

Rh{t)uh{t)  4-  X((V*1a4/.a](0-®a(0 

\fh{tfth,t  +  h)*Gh  f  f'  fh(tfthlTi  +  h)Bh(ri)uh{r})dri 
I  J\»/h\h 


(2.22)  (£»,.«„)(  <)  =  •  +  + 

•  /  Th{ri,T  +  h)Bh{T)uh(r)dr  drj\  , 

J 

For  4  £  [tf^  —  h,tfth]  ■ 

.  (^/i,iu/i)(4/,/i  —  h), 

when  s  £  [0,4//,)  and  by  (£v«uft)(4)  =  Rh{t)uh(t),  0  <  4  <  4 />H,  when  s  €  [4/^,4/],  for 
Uh  £  Uh-  We  again  have  that  for  j  =  0, 1,2,  •  •  • , kfth  -  1,  A;  =  j,j  -hi,*--  ,  A/,*  -  1,  and 

uhj  £  U h,j  i 


(2.23) 


{Rh,,Uh){t)  =  (Rh,jUh,j){k) 


for  s  £  ( jh,(j  +  l)h)  and  4  £  [Ah,  (A:  4-  l)h),  where  uh  £  Uh  is  given  by  (2.21)  and 
Rh,j  £  L(Uh,j)  is  given  by  (2.12).  We  note  also  that  Rh,»  is  self-adjoint  and  positive 
definite  on  Uh  and  that  if  we  let  Uhj  denote  the  subspace  of  Uh  obtained  from  Uh,:  via 
the  natural  embedding  (i.e.  via  (2.21)),  then,  Rh,»  is  a  bijection  from  Uhj  onto  Uhj- 
It  follows  therefore  from  (2.17),  (2.19),  (2.22)  that  for  j  —  0, 1,  2,  •  •  • ,  A/,a  -  1 


(2.24) 


&hj%hj8hj4>  ~  Bh^R-h^h,^ 


for  each  <j>  E  H  and  all  5  €  [jh,  ( j  +  l)/i),  and  that 

(2.25)  B'h,,Z^As<t>  =  0 
for  all  s  € 

Setting  11^(6)  =  Uh(k),  kh  <  s  <  (k  +  1  )h,  for  s  €  [0, £ /] ,  from  (2.11),  (2.12), 
(2.13),  (2.14),  and  (2.24)  we  find  that 

(2.26)  flh(s)<t>  =  fh(tfih,syGhfh(tf'h,s)4 

+  [tf'h  fh(t,sYQk(t)fh(t,s)Ut  -  KtSZ~h\Bh',<t> 

J  j»/Hjh 

for  each  <j>  E  H .  Note  that  (2.25)  implies  that  ri>,(£)  —  Gh  for  t  E  tf}< 

For  8,  E  L(H,Ua ),  Z,  E  L{U,),_ and  8’  E  L(Ua,H)  given  by  (2.2),  (2.3),  and  (2.4), 
respectively,  define  B,  E  L(H,U),  Za  E  L(U),  and  S*  €  L{U,H)  by 


(2.27) 

dm)  =  j 

l  0, 

l  (B.*)(0, 

0  <t<s, 
s  <  t  <  tf, 

(2.28) 

II 

■40 

'S' 

•» 

f  tf(Ou(f)» 
i  (£*u)(0, 

0  <  t  <  s, 
s  <t  <  tf 

and 

(2.29) 

s>  =  B,U 

for  4>  E  H  and  u  €  U.  It  is  not  difficult  to  show  that  8,  =  ( B ,)"  (i.e.  that  B,  E  L(U,H) 
is  the  Hilbert  space  adjoint  of  Ba  E  L(H,U)),  that  Z,  is  self-adjoint  positive  definite  on 
U,  and  that  if  we  let  U ,  denote  the  subspace  of  U  obtained  via  the  natural  embedding 
of  Ua  into  U,  then  Z,  is  a  bijection  from  Ua  onto  Ua.  Consequently,  if  follows  that 

(2-30)  = 

for  all  4>  E  H  and  s  E  [0,</].  From  (2.5)  we  obtain  that 

(2.31)  n(s)f>  =  T(t,,syGT(t,,s)<t>  + J  ’ T(t,s)-Q(t)T(t,s)<t>dt 

for  all  <f>  E  H  and  s  E  [0,  tf]. 

Our  fundamental  convergence  or  approximation  result  for  the  finite  time  horizon 
problem  is  given  in  the  following  theorem  and  its  corollary. 

THEOREM  2.4.  Suppose  that  the  families  of  operators  {T,B,Q,R}  satisfy  con¬ 
ditions  (Cl)-(C3)  and  that  for  all  h  >  0,  the  families  of  operators  {Th,  Bh,Qh,  Rh} 
satisfy  conditions  (Dl)  and  (D2).  Suppose  further  that  the  approximation  assumptions 
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(Al)-(A4)  are  satisfied.  Then  the  discrete  time  family  of  operators  II/,  =  {II/i(A:)  :  0  < 
k  <  kf,h}  given  by  (2.14)  or  (2.15)  strongly  approximates  the  continuous  time  family 
of  operators  II  =  {11(f)  :  0  <  t  <  tf}  given  by  (2.5)  or  (2.6)  on  the  set  A(l,f/).  That 
is,  I!/,  —*•  IT  on  the  set  A(1  ,tf). 

Proof.  The  desired  result  will  follow  from  Theorem  2.3  if  we  can  argue  that 
lim/,_o+  flh{t)4>  =  II(f)0,  uniformly  in  t,  for  t  G  (0,  t/] ,  for  each  <t>  G  H,  where  II/,  and 
IT  are  given  by  (2.26)  and  (2.31),  respectively. 

From  assumption  (A3),  we  have  that  the  operators  are  bounded  uniformly 
in  h  >  0  and  5  €  [0,  i /].  Somewhat  technical,  but  rather  elementary  arguments  can 
be  used  to  show  that  8h,s4>  — *■  for  all  G  H,  £/,)aJP/,u  — *  Z,u  for  u  €  U,  and 

8^,PhU  — *  8* u.  for  u  G  U,  uniformly  in  s  for  s  €  [0, f/] ,  where  B/,,a,  8,,  £/,.,,  Z,,  8^, 
and  B;  are  given  by  (2.17),  (2.27),  (2.22),  (2.28),  (2.19),  and  (2.29)  respectively  (see 
Appendix  B).  This  together  with  the  identity 

z-^Ph  -  PhZ;1  =  z£{Zk..Pk  -  PkZ,}*;1 

yield  that  lim/,_0+  ^h,,Zf^s8>h,a4>  =  8aZjl8„<f>,  for  <f>  G  H,  uniformly  in  s  for  s  G  [0, < /]. 
The  desired  convergence  can  then  be  obtained  from  assumptions  (Al),  (A2),  (A4) 
and  equations  (2.26),  (2.31).  □ 

Let  F  =  (i?(f)  :  0  <  t  <  tf}  and  5  =  {S(t,s)  :  0  <  s  <  t  <  tf}  be  respectively  the 
continuous  time  families  of  optimal  closed-loop  feedback  gain  operators  and  optimal 
closed-loop  state  transition  operators  for  the  continuous  time  LQR  problem  (P).  That 
is,  for  t  €  [0,t/] 

F(t)  =  R(t)-'B(ty 11(f)  G  L{H,  U), 

and  for0<s<f<f/ 

(2.32)  S(t,s)<f>  =  T(t,s)<j>  -  J  T(t,rj)B(r])F('n)S(r},s)<i)dT] 

=  T(t,s)<f>-  f3T(t,r,)B{ri)(Z;l8;f>)(r1)dr], 

for  (j>  G  H  (see  [G]).  Similarly,  for  the  discrete  time  problem,  let  the  discrete  time 
families,  Fh  =  {Fh{k)  :0<k<  JfcM  -  1}  C  L{H,  U)  and  Sh  =  {Sh{k,j)  :  0  <j<k< 
fc/,/,}  C  L(H)  be  given  by 

Fh(k)  =  Rh(k)-lBh(kyUh(k  +  1  )Ah(k), 

where 

Rh{k)  =  Rh(k)  +  hBk(kyUh(k  +  l)Bh(k), 


k  =  0, 1,  •  •  • ,  kfih  -  1,  and 

*- 1 

(2.33)  Sh(k,j)4>  =  Th{k,j)(f>  -  h  ^  Th(k,i  +  l)Bh{i)Fh[i)Sh{i,j)<i> 

*=i 
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0  <  j  <  k  <  kfih ,  for  <j>  €  H. 


COROLLARY  2.1.  Suppose  that  the  hypotheses  of  Theorem  2.4  above  are  satis¬ 
fied  and  let  {u,x}  and  {uh,xh}  be  the  optimal  control/trajectory  pairs  for  the  LQR 
problems  (P)  and  (P*),  respectively,  corresponding  to  the  initial  data  x(0)  =  x/,(0)  = 
xo  6  H .  Then 

fi)  -  F; 

(ii)  Sh  ->  S; 

(Hi)  limh_0+  ||«fc(A:fc)-«(t)||u  =0,  and  limfc^0+  ||xa(At/i) -x(t)\\H  =  0,fort  e  [0,f/] 
and  for  all  nets  {kh}h> o  for  which  lim/,_o+  =  t. 

(iv)  lixn/,_*0+  Jh  =  J- 

Proof.  Statement  (i)  and  (iv)  (recall  (2.7)  and  (2.16))  are  immediate  consequences 
of  Theorem  2.4.  Statement  (iii)  follows  from  statement  (i)  and  (ii)  since  u(t)  = 
-F(t)x(t),  x(t)  =  S(t,  0)x0,  t  €  [0 ,tf],  and  uh(k)  =  -Fh(k)xh(k),  0  <  k  <  kfih  -  1, 
Xh(k)  =  Sf,(k, 0)xo,  0  <  k  <  kf:h-  Thus  we  need  only  to  verify  statement  (ii). 

We  rewrite  (2.32)  as 

S(t,  s)<fi  =  T(t,  s)<t>  -  £  T{t ,  v)B(r})  (Zj'Bft)  ( v)dr, , 
and  from  (2.33)  we  obtain 

S„(t,s)<f,  =  T„(t,s)4.  -  I  TH(t,n  +  h)Bk(n)(^BlMn)dn- 

J[s/h\h 

The  result  now  follows  as  in  the  proof  of  Theorem  2.4  □ 

REMARK  In  actual  practice,  given  the  continuous  time  LQR  problem  (P),  the 
net  of  discrete  time  problems  {(P/,)}  is  typically  obtained  by  considering  zero-order 
hold  (i.e.  piecewise  constant)  control  inputs  and  output  sampling.  In  this  case  we 
would  obtain  Ah{k)  =  T{{k  +  l)h,kh),  Bh{k)  =  h'1  ftf+1)h  T{{k  +  1  )h,s)B{s)ds, 
Qh{k )  =  h~x  fth+1'h  Q[s)ds,  Rh(k )  =  h~l  f^+1^h  R(s)ds,  and  Gh  =  G.  When  con¬ 
ditions  (Cl)-(C3)  on  the  continuous  time  families  T,B,Q,  and  R  are  satisfied,  it  is 
immediately  clear  that  the  discrete  time  families  Th,Bh,Qh  and  Rh,  and  the  oper¬ 
ator  Gh  satisfy  conditions  (Dl)  and  (D2)  and  the  approximation  conditions  (Al)- 
(A4).  More  generally,  other  discretizations  are  also  admissible.  For  example,  in  the 
time  invariant  case,  the  semigroup  {T(t)  :  t  >  0}  could  be  discretely  approximated 
using  A-stable  Pade  approximants  to  the  exponential  (see  [HK]).  In  particular,  if 
T{t)  =  exp(M),  t  >  0,  then  one  might  set  7\(A:)  =  (/  -  hA)~k  (implicit  Euler)  or 
Th(k)  =  (/  —  hA/2)~k(I  +  hA/2)k  (Crank-Nicolson).  The  stability  and  consistency 
of  these  discretizations  (i.e.  assumption  (A2))  can  be  verified  using  the  theory  and 
techniques  developed  in  [HK].  Finally,  along  these  same  lines,  the  convergence  of  si¬ 
multaneous  but  independent  state  and  time  discretization  in  the  context  of  the  LQR 
theory  should  also  be  looked  at.  We  note  that  appropriately  “coupled”  state  and 
time  discretization  can  be  handled  using  the  theory  and  framework  which  has  been 
developed  above. 
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3.  The  Infinite  Time  Horizon  Problem.  The  linear  quadratic  regulator  prob¬ 
lem  over  an  infinite  time  interval  can  be  viewed  as  an  extension  of  the  finite  time 
interval  problem.  The  state  equations  (2.1)  and  (2.9)  governing  the  dynamics  of  the 
continuous  time  and  discrete  time  control  systems,  respectively,  remain  the  same.  The 
continuous  and  discrete  time  operator  families  {T,  B,Q,R},  and  {2\,  B^,Qh,  Rh}  are 
assumed  to  be  defined  on  the  infinite  time  intervals  [f0>+oo)  C  R  and  [fc0,+oo)  C  Z, 
respectively.  The  cost  functionals  are  taken  to  be 

(3.1)  Joo(u;to,z(t 0))  =  [  {<  Q{t)x(t),x(t)  >H  +  <  R{t)u(t),u(t)  >v}dt 

=  lim  J(u-,to,x(to),  0) 

tf— >00 

and 

00 

Jh,oo{uh;ko,xh(k0))  =  h  YA<  Qh{k)xh(k),xh{k)  >h  +  <  Rh(k)uh(k),uh(k)  >v } 

k=ko 

(3.2)  =  lim  Jh{uh\k0,xh(k0),0) 

kf—*00 

Under  appropriate  stabilizability  and  detectability  assumptions  on  the  continu¬ 
ous  time  and  the  discrete  time  control  systems,  the  existence  and  the  uniqueness  of 
the  optimal  controls  u,uh  minimizing  (3.1)  and  (3.2),  respectively,  can  be  obtained. 
Moreover,  these  optimal  controls  can  be  written  in  a  closed-loop  state  feedback  form 
(see  Theorem  3.1  below).  We  are  again  interested  in  investigating  the  convergence  of 
the  optimal  controls  and  the  optimal  feedback  laws  for  the  sampled  systems  as  the 
length  of  the  sampling  interval  tends  toward  zero. 

Our  fundamental  result  can  be  outlined  as  follows.  Assume  that  the  conditions 
(Al)-(A4)  for  the  convergence  of  the  open-loop  control  problems  are  satisfied  on 
every  finite  time  interval  [t0,t/].  Suppose  further  that  the  stabilizability  and  the 
detectability  of  the  continuous  time  system  are  uniformly  preserved  by  the  sampled 
time  systems  (see  Definition  3.3-(iii)  and  3.4-(iii)).  Then  the  optimal  controls  u * 
and  the  optimal  state  feedback  laws  F for  the  sampled  time  systems  converge  to 
the  optimal  control  u  and  optimal  feedback  law  F  for  the  continuous  time  system, 
respectively,  as  the  length,  h,  of  the  sampling  interval  tends  toward  zero.  We  note 
that  the  problem  of  uniform  preservation  of  stabilizability  and  detectability  under 
sampling  is  in  general,  a  difficult  one.  Here  we  shall  treat  this  question  only  in  a 
limited  sense.  We  shall  have  to  assume  finite  rank  feedback  stabilizability  and  finite 
rank  detectability  (see  Condition  (F))  for  the  continuous  time  system,  although  we 
have  some  conjectures  about  other  reasonably  broad  classes  of  systems  for  which  these 
conditions  can  be  verified.  We  address  this  question  in  greater  detail  below. 

As  in  the  finite  time  horizon  problems,  the  functionals  and  Jh  oo  can  be  viewed 
as  quadratic  forms  on  L2(to>oo;U)  and  /2(fc0,  oo;  U),  respectively.  However,  since  J & 
and  Jh i00  are,  in  general,  not  bounded  (for  example,  the  uncontrolled  system  may 
not  be  asymptotically  stable,  hence  the  cost  for  the  control  input  u  =  0  may  be 
infinity),  one  must  deal  with  some  rather  tedious  technical  details.  Therefore,  the 
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infinite  horizon  LQR  problem  is  commonly  viewed  as  the  asymptotic  limit  of  the 
finite  time  horizon  problems  with  the  final  time,  t /,  tending  to  infinity.  The  existence 
and  the  uniqueness  of  the  optimal  controls  and  feedback  laws  can  then  be  obtained 
by  considering  the  limiting  behavior  of  the  optimal  controls  and  the  optimal  feedback 
laws  for  the  finite  time  horizon  problems.  From  the  definitions  of  the  functionals  J 
and  Jh, oo,  it  is  natural  to  view  the  infinite  time  horizon  problem  as  the  limit  of  the 
finite  horizon  problems  with  the  final  state  penalty  operators  G  and  G k  taken  to  be 
zero.  However,  we  note  that  if  the  optimal  trajectories  z(t)  and  Xh(k)  are  known  to 
be  asymptotically  stable  (i.e.,  x[t)  — >  0,  xh{k )  — ►  0,  as  t  and  k  tend  to  infinity),  then 
the  infinite  horizon  LQR  problem  can  also  be  considered  as  the  limit  of  finite  horizon 
problems  with  Gk,  G  >  0.  Once  again,  for  simplicity,  we  shall  assume  henceforth, 
without  lost  of  generality,  that  tQ  =  k0  =  0. 

DEFINITION  3.1.  (Cost  functional  stabilizability) 

(i)  The  continuous  time  system  associated  with  the  operator  pair  {T,  B}  is  said  to 

be  cost  functional  stabilizable  with  respect  to  the  performance  index  Jqo  given 
by  (3.1),  if  for  each  <f>  £  H,  there  exists  a  constant  M(<f> )  such  that  for  any 
s  >  0,  there  exists  a  control  input  u,  £  L2(S)  oo;  U )  with  s,  4> )  < 

(ii)  The  sampled  time  system  associated  with  the  operator  pair  {7\,  Bh}  is  said  to 
be  cost  functional  stabilizable  with  respect  to  the  discrete  performance  index 
Jh, oo  given  by  (3.2),  if  for  each  <t>  £  H,  there  exists  a  constant  M/,(<£)  such 
that  for  any  j  >  0,  there  exists  a  control  input  sequence  uhj  £  l2(j,oo-,U) 
with  Jhi00(uhij;j\<t>)  <  Mh(4>). 

(iii)  The  sampled  systems  are  said  to  be  uniformly  cost  functional  stabilizable  for 
all  0  <  h  <  h0,  if.  for  each  <f>  £  H,  the  constants  Mh((j>)  defined  in  (ii)  are 
independent  of  the  length  of  the  sampling  interval  h ,  for  all  h  <  h0  for  some 
Hq  >  0. 

For  any  given  final  time  t /  and  final  index  kfih,  let  n(/(-;G)  and  Tlkf h(-\Gh) 
denote  the  Riccati  operators  given  by  (2.5)  and  (2.14)  corresponding  to  the  final  state 
penalty  operators  G  and  Gk,  respectively.  In  the  case  G  =  Gk  =  0,  using  (2.7)  and 
(2.16),  it  is  easy  to  verify  that  (see  for  example,  [DI])  for  each  given  t  >  0  and  k  >  0, 
the  functions  t /  i — *•  nt/(t;0)  and  k/^  h[k\0)  are  nondecreasing,  self-adjoint, 

nonnegative  operator  valued  functions.  The  assumed  cost  functional  stabilizability  of 
the  continuous  and  discrete  time  control  systems  then  provides  an  upper  bound  for 
nt/  and  Hhit/h.  Indeed,  we  have 

<  nt/(t;0)<M  >#5:  <f)  £  H, 

and 


<  >H<  Mh(<f>),  4>£H, 

for  all  tf  and  kfih.  Thus,  the  strong  limits  of  nt/(t;0)  and  Ylh,kf_k{^'i0)  exist  for  each 
t  >  0  and  k  >  0  as  tf  and  kfh  tend  to  infinity.  Let  us  denote  these  strong  limiting, 
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operator  valued  functions  by  IToo(-;0)  and  n>i|OO(-;0),  respectively.  The  existence  and 
uniqueness  of  the  solutions  to  the  continuous  and  discrete  time  optimal  control  prob¬ 
lems  is  given  in  the  following  well  known  theorem;  see,  for  example,  [BW],  [G],  [GRJ, 
[LCB],  [HH],  and  [Z]. 

THEOREM  3.1.  Assume  that  the  continuous  time  system  and  the  sampled  time 
systems  for  all  h  sufficiently  small  are  cost  functional  stabilizable.  Then  for  any  s  >  0 
and  j  >  0,  and  initial  states  x(s)  =  <j>  and  Xj  ^  =  <j>,  there  exist  unique  optimal  controls 
u,  and  uh  which  minimize  the  cost  functionals  Joo(-\s,x(s)\0)  over  L2(s,  oo;  £/)  and 
Jk,oa{'\  j,  Xh(j)‘,  0)  over  l2 (j,oc\U),  respectively.  The  optimal  controls  can  be  written 
in  linear  state  feedback  form  as 

u(t)  =  -Rity'Bitynnfcowt)  =  -F(t)x(t), 


and 


uh(k)  =  -  Rh(k)-1  Bh(kyilh,Qo(k  +  1;0  )Ah(k)xh(k)  =  -Fh(k)xh(k), 

where  x  and  xh  are  the  corresponding  optimal  trajectories  and  Rh{k)  =  Rh(k)  + 
hBh(k)'Tlh,oo{k  -f  "•O)Bh.(k),  The  operator-valued  function  n<x>(-;0)  is  bounded  on 
the  interval  [0,  oo)  and  satisfies  the  Riccati  integral  equation 

(3.3)  R^s-O)^  =  T{t,syn„{t;0)T{tta)<f> 

F(t,s)*\Q{t)  —  noo(r;0)(BR~1B*)(r)IIoo(r;  0)]T(r,  s)<f>dr, 

for  all  (j)  €  H  and  ( t,s )  €  A(2,oo).  Similarly,  the  operator-valued  sequence  I"Ui0O(-;0) 
is  bounded  for  0  <  k  <  oo  and  satisfies  the  Riccati  difference  equation 

(3.4)  n,>oo(A:;0)  =  Ah{kyjlhi0O{k  +  1;  O)^*)  +  hQk{k) 

-hAh(kynhi(X1{k  +  l;0)Bh{k)Rh{k)-1Bh{kyUhtOO{k  +  l-0)Ah{k). 

If  the  sampled  time  systems  are  uniformly  cost  functional  stabilizable  for  0  <  h  <  h0, 
then  the  operator-valued  sequences  ri/,)CO(-;0)  are  uniformly  bounded  for  all  sampling 
period  h  with  0  <  h  <  h^. 

We  assume  that  the  general  conditions  (A1)-(A4)  for  the  convergence  of  the 
open  loop  problems  hold  on  any  given  finite  time  interval.  From  the  feedback  form 
of  the  optimal  controls  given  in  Theorem  3.1,  it  is  not  difficult  to  see  that  on  a 
given  finite  time  interval  [0,f/|,  the  uniform  convergence  of  the  optimal  controls  u^, 
the  optimal  trajectories  x and  the  optimal  feedback  gains  Fh  for  the  sampled  time 
control  problems  would  follow  directly  from  the  uniform  convergence  of  n*iO0, (•;  0) . 
Our  investigation  is  therefore,  focused  on  the  convergence  of  n^  oo(-;0)  to  II0o(-;0)  as 
h  tends  toward  zero.  Using  the  notation  introduced  in  the  previous  section,  we  note 
that  for  each  t  >  0,  an  obvious  sufficient  condition  for  the  convergence  of  ri/,>0o(t;C)) 
to  1130(4;  0)  is  the  convergence  of  11^(4;  G)  to  rioo(4;0)  and  the  uniform  convergence 
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in  h  of  ft h,t/A(t;Gh)  to  nh,oo(t;0)  (with  k/h  =  \tf/h ])  as  tf  tends  to  infinity  for  some 
G  >  0  and  corresponding  G >,  >  0.  Indeed,  from  the  triangle  inequality,  for  4>  €  H,  we 
have 

||n^,oo(i;o)d>  -  iioofcoMi*  <  ||nfcloo(«;o)^  -  gh)4>\\h 

+Pmm(*;  Gk)*  -  n ,,(*;  d)4>\\a  +  ||nt/(t;  G)4>  -  n«(t;0)^||H. 


Then  for  an  arbitrary  e  >  0,  a  sufficiently  large  tf  can  be  chosen  such  that  the  first 
and  the  last  terms  on  the  right  hand  side  of  the  above  inequality  are  smaller  than  e/3 
for  all  h.  By  applying  the  theory  of  the  previous  section  on  the  interval  [0,i/],  there 
exists  h0  >  0  small  enough' such  that  for  all  0  <  h  <  /to,  the  second  term  on  the  right 
hand  side  of  the  above  inequality  is  bounded  by  e/3.  Thus,  the  desired  convergence 
immediately  follows. 

As  we  have  pointed  out,  if  the  trajectories  of  the  systems  are  asymptotically  stable, 
then  as  tf  tends  to  infinity,  the  cost  functionals  J ^  and  Jhi00  are  also  limits  of  the  cost 
functionals  J,  Jh  for  the  finite  time  interval  problems  on  [0,  tf]  with  final  state  penalties 
G  and  Gh  different  from  zero.  In  particular,  if  the  optimal  trajectory  of  the  infinite 
horizon  problem  is  asymptotically  stable,  the  convergence  rates  of  J/,(uh  :  k,  <f>,  Gh)  — < 
>H  with  Gh  >  Mh[<j>)  and  J[u;t,d>,G)  =<  II tf(t;G)<j>,<f>  >H  with 
G  >  M{4>)  can  be  estimated  by  the  decay  rate  of  the  optimal  trajectory  x  for  the 
infinite  horizon  problem.  Toward  this  end,  let  S  =  (S(t,s)  :  0  <  s  <  t  <  oo}  be  the 
continuous  time  evolution  system  given  by 

(3.5)  S(t,s)<j)  =  T{t,s)<j>  —  J'  T(t,T)B(T)F{r)S(T,s)<t)ds,  for  <f>  E  H. 

The  evolution  system  S  is  also  referred  to  as  the  perturbation  of  T  by  — BF .  It 
is  not  difficult  to  verify  that  S(t, 0)0  corresponds  to  the  optimal  trajectory  for  the 
continuous  time  infinite  horizon  problem  with  initial  state  <f>  €  H.  Similarly,  let  the 
discrete  time  evolution  system  S*  =  {Sh(i,j)  :  0  <  j  <  i  <  oo}  be  defined  as 


(3 .6)Sh{i,j)  = 


*  = 


n i-JjiMk)  -  hBHWR^ky'BWn^ik  +  l;0)Ak(*)},  *  >  j. 


Thus,  5/,  (k,  0)</>  is  the  optimal  trajectory  for  the  discrete  time  infinite  horizon  problem 
with  initial  state  4>  6  H. 

DEFINITION  3.2.  (Exponential  stability  of  the  optimal  feedback  systems) 

(i)  The  optimal  continuous  time  feedback  system  (3.5)  is  said  to  be  exponentially 
stable,  if  there  exist  constants  M  and  a  >  0  such  that  for  all  0  <  s  <  t  <  oo, 
l|S(M)IU(if)  <  M exp{— a(f  -  s)}. 

(ii)  The  discrete  time  optimal  feedback  system  (3.6)  is  said  to  be  exponentially 
stable,  if  there  exist  constants  M *  and  ah  >  0  such  that,  for  all  0  <  j  <  i  <  oo. 
l|Sk(«\j')l|i(*)  <  Mhexv{-ah(i  -  j)h}. 


(iii)  The  sampled  time  optimal  feedback  systems  are  said  to  be  uniformly  expo¬ 
nentially  stable  for  all  0  <  h  <  h0,  if  the  constants  M h  and  >  0  in  (ii) 
above  are  independent  of  h  for  0  <  h  <  ho. 

The  following  result  is  an  important  property  of  the  solutions  of  the  Riccati  equa¬ 
tions  on  the  infinite  time  interval  when  the  optimal  feedback  systems  are  exponentially 
stable.  The  proof  can  be  found  in  [BW],  [DI],  [G]  for  the  continuous  time  problem, 
and  in  [GRJ  (Theorem  2.9)  for  the  discrete  time  problem. 

LEMMA  3.1.  Assume  that  the  continuous  time  control  system  and  the  sampled 
time  control  system  with  sampling  period  h  are  cost  functional  stabilizable.  If  the 
corresponding  optimal  feedback  systems  are  exponentially  stable,  then  IToo^O),  and 
BU.oolsO)  the  unique  bounded  solutions  of  the  corresponding  Riccati  equations 
(3.3)  and  (3.4)  on  the  infinite  time  interval.  Furthermore,  if  G  and  Gh  are  chosen 
such  that  G  >  TIoo^O)  and  Gh  >  Tlh  oo(k-,0)  for  all  t  and  k,  then  the  solutions  of  the 
Riccati  equations  on  the  fitiite  time  interval,  ITt/(t;G)  and  Tlh^, K{k‘,Gh),  satisfy 

<  nt/(£;  G)<f>  —  noo(t;  0)<j>,  <j>  >jj  <  <  GS(tf,t)<f>,  S(tf,t)<j>  >n, 

and 

<  n/i,jfc/A(A:;  Gh)<f>  —  nhoo(fc;  Gh)(f>,<f>  >r  <  <  GhSh(kfh,k)<f),Sh{kf:h,k)(i>  >#, 

respectively,  for  all  t  <tj,  k  <  kfth,  and  <f>  €  H. 

LEMMA  3.2.  Assume  that  the  sampled  systems  are  uniformly  cost  functional 
stabilizable  with  the  optimal  feedback  systems  uniformly  exponentially  stable  for  0  < 
h  <  ho.  Then,  the  operators  G  and  Gh  can  be  chosen  as  described  in  Lemma  S.l  with 
Gh  <  C  ■  I  for  some  constant  C  independent  of  h.  As  tf  tends  to  infinity,  IIt/(-;G) 
converges  to  n^sO)  uniformly  on  any  bounded  subinterval  [ a,b ]  of  [0,  oo)  and  the 
convergence  of  */,*(■;  Gh)  with  kjh  —  [tf/h]  to  nhjOO(-;0)  is  uniform  in  h  for  all 
0  <  h  <  h0  on  any  bounded  subinterval  [a,  b ]  of  [0,  oo)  in  the  uniform  operator  norm. 

Proof.  We  prove  only  the  discrete  time  assertion.  The  continuous  time  case 
is  completely  analogous,  if  not  simpler.  The  assumption  of  uniform  cost  functional 
stabilizability  implies  that  the  operators  Gh  can  be  chosen  as  stated  in  the  theorem. 
Then  let  M  and  a  be  the  constants  in  Definition  3.2-(iii).  For  a  given  e  >  0  and 
t  6  [a,  6],  we  can  take  tf  large  enough  such  that  CM2  exp{— 2a(t/  -  t  —  h0 )}  <  e.  Let 
kh  =  \t/h],  then  ( kf  ^  —  kh)h  >  tf  -  t  —  h0  for  all  0  <  h  <  ho.  Since  Gh)  > 

rL)Oo(kfc;0),  we  have 

~  %<»(<; 0)|U(H)  =  l|n*,*M(A:h;C7fc)  -  nhoo(A:k; 0) ||z,(/r) 

sup  <  (T\.htkf  hikh'i  G h)  ~  n^,oo(k^;  O))0,  <f>  >// 
W»<i 

<  sup  <GhSh{kfth,kh)<f>,Sh{kfth,kh)4>>H 
Mw<i 

<  CM2e~2a{kf-k~kh)h  <  e. 
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□ 

As  a  consequence  of  the  above  lemma,  we  obtain  our  first  convergence  result. 

THEOREM  3.2.  Let  Conditions  (Al)-(A4)  for  the  operator  families  {3\,  B *,  Qh,  i?h} 
hold  on  any  finite  subinterval  of  (0,  oo).  Assume  further  that  the  continuous  time  sys¬ 
tem  and  the  sampled  time  systems  with  0  <  h  <  h0  are  uniformly  cost  functional 
stabilizable,  and  that  the  optimal  closed-loop  evolution  systems  are  uniformly  expo¬ 
nentially  stable  .  Then,  the  Riccati  operators  ri/,iOO(£;0)  converge  strongly  to  IIoo(t;0) 
and  the  convergence  is  uniform  on  any  bounded  subinterval  of  [0,  oo). 

Proof.  Let  <f>  €  H  and  let  (a,  b]  be  a  bounded  subinterval  of  [0,oo).  We  choose 
an  operator  G  such  that  G  >  IIoo^O)  and  G  >  IT^oo (&;0)  for  all  t  £  [0,  oo)  C  R, 
k  £  [0,  oo)  C  Z  and  0  <  h  <  hQ.  By  Lemma  3.2,  tf  can  be  taken  large  enough  such 
that  for  all  kjh  =  [tj/h\,  we  have 

||nt/(t;G)d>-noo(f;0)*<|  and  ||nM/ik(t;  G)4>  -  flhtOO{t-,0)<f>\\H  <  |, 

for  all  t  £  [a,  6]  and  all  0  <  h  <  h0.  By  Theorem  2.4  of  Section  2,  we  can  find  h  small 
enough  such  that 

for  all  t  £  [a,b\.  Therefore,  we  have 

||n/l,oO(«;0)^-noo(f;0)<^||H  <  ||nfc.oo(^o)0-nh,fcM(t;G)^||H 

+  ||nM/,(t;  G)4>  -  nt/(t;  G)4>\\h  +  ||nt/(t;  G)4>  -  LU*; 0)tf>||*  <  e, 

for  all  t  £  [a, b}.  □ 

We  note  that  although  the  exponential  stability  of  the  optimal  feedback  systems  is 
only  a  sufficient  condition  for  the  uniqueness  of  the  solutions  to  the  Riccati  equations, 
it  also  implies  the  stability  of  the  solutions  to  the  Riccati  equations  under  small 
perturbations  (see  [BW],  [DI])  which  is  important  in  the  context  of  approximation. 
Consequently  the  remainder  of  our  discussions  here  are  concerned  with  conditions 
which  guarantee  the  exponential  stability  and  uniform  exponential  stability  of  the 
optimal  feedback  systems. 

A  useful  characterization  of  exponentially  stable  evolution  systems  is  given  in  a 
result  due  to  Datko  in  the  continuous  time  case  (see  [D])  and  Zabczyk  in  the  discrete 
time  case  (see  [Z]).  We  state  it  here  in  both  its  continuous  and  discrete  time  forms  as 
a  lemma. 

Lemma  3.3. 

(i)  Let  T  be  a  strongly  continuous  evolution  system.  If  there  exists  constants 
Ci,C2,  and  u>  >  0  such  that 

llrlMjIUm  <  and  ff  || T[t,*WBdt  <  C,U lit. 
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for  all  4>  (E  H  and  0  <  s  <  t  <  oo,  then,  we  can  find  constants  M  and  a  >  0, 
depending  only  onCi,Ci,  andu,  such  that  ||T'(^,  s) [|x,(jy)  <  M exp{— a(t~ 5)}, 
for  all  0  <  s  <  t  <  00. 

(it)  Let  Th  be  the  discrete  time  evolution  system  defined  by 


Th(i,j)  = 


I,  *  =  J, 

nr4  Ah(k),  *  >  j- 


If  there  exist  constants  and  C 2i/,  such  that 


l|Tk(.,j)|U(„,  <  and  k  £ \\%, 

i~k 

for  all  0  <  k  <  00  and  d>  G  H ,  then,  we  can  find  constants  Mf,  and  ah  >  0, 
depending  only  on  Ci^^h  and  C2>,  such  that  for  all  0  <  j  <  i  <  00 

||n(t,j)||iW  <  Mhe-a«-»k. 


The  converse  of  this  lemma  is  obviously  true.  By  the  uniform  cost  functional 
stabilizability,  the  solutions  of  the  Riccati  equations  (3.3)  and  (3.4),  Iloo  and  n*i00  are 
uniformly  bounded,  and  therefore,  the  evolution  systems  S  and  Sk  given  by  (3.5)  and 
(3.6),  respectively,  are  uniformly  exponentially  bounded  if  T  and  Th  are  uniformly 
exponentially  bounded.  Moreover,  we  have 

ff  <  M\\m 

and 

'»£iw»(*)1/,s>(M*IIh  <  mm*, 

k=j 

for  some  constant  M  and  for  all  <f>  €  H.  If  the  operators  Q(t)  and  Qh{k)  are  uniformly 
strictly  coercive  (i.e.,  there  exists  a  constant  q  >  0,  such  that  <?(t)  >  ql  and  Qh[k)  > 
ql ,  for  t  >  0  and  k  >  0),  we  can  immediately  rriclude  that  S  and  are  uniformly 
exponentially  stable. 

A  more  general  case  in  which  the  boundedness  of  the  cost  functional  implies  the 
stability  of  the  feedback  system,  is  the  case  of  detectable  systems. 

DEFINITION  3.3.  (Detectability) 

(i)  A  continuous  time  control  system  is  said  to  be  detectable  with  respect  to 
the  cost  functional  (3.1),  if  there  exists  a  bounded  operator-valued  function 
Vr(-)  :  [0, 00)  t-4  L(H)  such  that  the  evolution  system  TV,  corresponding  to 
the  perturbation  of  T  b yVQl^2(-),  is  exponentially  stable. 

(ii)  A  sampled  time  control  system  is  said  to  be  detectable  with  respect  to  the  cost 
functional  (3.2),  if  there  exists  a  bounded  sequence  of  operators  { V), (/c) }*L0  C 
L(H)  such  that  the  discrete  time  evolution  system  Tv,h  given  by 

I,  i  =  J, 

n \Z)(Mk)  +  kVh(k)Qh(k)1/2),  i  >  j , 


Tv,h{i,j)  = 
is  exponentially  stable. 
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(iii)  The  sampled  time  systems  arp  said  to  be  uniformly  detectable  for  0  <  h  <  h0, 
if  there  exist  constants  Ci,Cz,  and  a  >  0,  independent  of  h  such  that  the 
operator-valued  sequences  {I4(A:)}£10  in  (ii)  satisfy  ||Vfc(fc)||i,(.ff)  <  C\  and 

||rvik(*,i)|U(H)  <  C2e~a^\  0  <  j  <  i  <  oo, 

for  all  sampling  rates  0  <  h  <  ho. 

Under  appropriate  conditions,  the  detectability  of  the  control  systems  implies  the 
stability  of  the  optimal  feedback  systems.  Indeed,  toward  this  end,  we  require  the 
following  boundedness  assumption. 

(B)  The  continuous  time  evolution  system  T  and  the  discrete  time  evolution  system 
Tk  are  uniformly  exponentially  bounded  on  A(2,oo).  That  is,  there  exist 
constants  M  and  oj  such  that 

for  0  <  s  <  t  <  oo  and  0  <  j  <  i  <  oo.  The  operator  families  B,Q,  and 
R  and  the  piecewise  constant  operator  families  Bh,Qh,  and  Rh  are  uniformly 
bounded  in  norm  by  a  given  constant  C  on  the  entire  interval  [0,oo)  for  all 
sampling  rates  h  >  0.  Furthermore,  there  exists  a  constant  r  >  0  such  that 
R(t)  >  rl  and  Rh{t)  >  rl  for  all  t  >  0,  and  h  >  0. 

THEOREM  3.3.  Consider  a  detectable  continuous  time  control  system  and  a 
detectable  sampled  time  system  which  are  both  cost  functional  stabilizable.  Assume 
that  the  evolution  systems  T ,  Th  are  exponentially  bounded,  and  the  operator  families 
{B,Q,R}  and  {Bh,Qh,  Rh}  wz  bounded  in  norm  on  the  infinite  time  interval.  Then, 
the  optimal  feedback  systems  for  both  systems  are  exponentially  stable.  Furthermore, 
suppose  that  constants  C,u>,r  >  0,  and  a  >  0  can  be  found  such  that  the  following 
conditions  are  satisfied. 

(i)  The  operator  families  {B,  Q,  R,  IIo^-;  0),  V}  and  {Bh,  Qh,  R^,  rUiOO(-;0),  Vh} 
are  bounded  in  norm  by  C ; 

(ii)  For  all  t  >  0,k  >  0,  R(t)  >  rl  and  Rh  >  rl; 

(iii)  The  evolution  systems  T,Th,Tv,  and  Ty,h  satisfy 

!|r(f,s)||t(»|  <  ||Tv(e,s)!U,„,  < 

and 

lin(«,j)!U;*,  <  ||7V(i,j)Hi(ir)  <  Ct- 

Then  there  exists  constants  M  and  (3  >  0  depending  only  on  the  constants  C ,  r,  a  and 
w  such  that 

l|s(t,.)||1(»,  <  Me-*'-',  ||s*(«-,j)|U,w,  <  Mr'i"11*. 
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Moreover,  under  Assumption  (B),  if  the  sampled  systems  are  uniformly  detectable  and 
uniformly  cost  functional  stabilizable  for  0  <  h  <  h0,  then,  the  optimal  closed-loop 
systems  are  uniformly  exponentially  stable  for  0  <  h  <  ho. 

P  '/.  In  the  case  of  continuous  time  system,  a  proof  is  given  by  Da  Prato  and 
Ichikmva  in  [DI].  The  dependence  of  the  exponential  bound  for  the  optimal  closed- 
loop  system  on  the  constants  indicated  above  is  proved  in  [W].  The  arguments  for  the 
discrete  time  case  are  very  similar  to  those  used  in  the  continuous  time  case.  Indeed, 
let  correspond  to  the  perturbation  of  Tyth  by  A*  =  {A k(k)  =  — Bk(k)Fh(k )  + 
V^(k )Qh(k)1/2}  in  the  sense  that 

-  -  rU/h)  -  - 

Sh{t,s)<f>  =  TVth[t,s)<f>  +  /  TVih{t,T)Ail{T)Sk(T,s)<t>dT. 

J[s/h\ 

Let  us  define 

/  (M)  =  -Rh{k)l/2Fh(k)<i),  and  gh{k,i)  =  Qh(k)1/2Sh(k,i)(f), 
for  k  >  i  >  0.  Then  cost  functional  stabiiizability  implies  that 

!!//,(•,  0  l|i3(*.oo;C/)  <  C\\<f>\\ H,  <  C\\4>\\h- 

The  evolution  system  Tv,h  is  bounded;  ||3V,A(f,i)IU(i?)  <  C  exp{  —  a(i  —  j)h}.  Thus  we 
obtain 

+  !|V’A(r)||£(H)||yh(r,«)||H)<ir, 

and  by  Young’s  inequality  (see,  [A,  Theorem  4.30,  p.90j),  we  have 

J™  \\sh(t,s)4>\\2Hdt  <  KU\\\,<t>eH, 

for  some  constant  K.  Applying  Lemma  3.3,  we  obtain  the  exponential  stability  of 
Sh-  The  dependence  of  the  exponential  bound  for  Sh  on  the  indicated  constants  of 
course  follows  from  the  dependence  of  the  constant  K  on  the  indicated  constants  as 
prescribed  in  the  lemma.  In  this  way  it  is  easy  to  see  how  under  Assumption  (B) ,  uni¬ 
form  detectability  and  cost  function  stabiiizability  will  imply  the  uniform  exponential 
stability  of  the  closed-loop  systems.  □ 

Another  closely  related  control  theoretic  concept  is  the  stabiiizability. 

DEFINITION  3.4.  (Stabiiizability) 

(i)  A  continuous  time  system  is  said  to  be  stabilizable,  if  there  exists  a  bounded 
operator-valued  function  K(-)  :  [t0,  oo)  >-+  L(H,  U)  such  that  the  evolution 
system  Tr  corresponding  to  the  perturbation  of  T  by  BK  is  exponentially 
stable. 
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(ii) 


A  sampled  system  is  said  to  be  stabilizable,  if  there  exists  a  bounded  sequence 
of  operators  {K\(fc)}JL0  C  L(H,  U)  such  that  the  discrete  evolution  operator 
Tk,h  given  by 


Tk,h^J)  = 


I,  *  =  J» 

nilrJ}(Mk)  +  kB^k)K^k))^  i>p 


is  exponentially  stable. 

(iii)  The  sampled  time  systems  for  are  said  to  be  uniformly  stabilizable  for  0  < 
h  <  ho  if  there  exist  constants  C\,C-i,a  >  0  independent  of  the  sampling 
period  h,  such  that  K ^  and  satisfy 

l|K*Wllw>  <  Cu  ||3V.*(i,»|Uw  <  c, «-*<'->■  >\ 

for  all  0  <  k  <  oo,  0  <  j  <  i  <  oo. 

Using  Theorem  3.3  it  is  easy  to  verify  that  cost  functional  stabilizability  and  de¬ 
tectability  imply  stabilizability  (take  K  =  F,  =  Fh,  for  example).  Conversely, 
stabilizability  clearly  implies  cost  functional  stabilizability.  Therefore,  under  the  uni¬ 
form  detectability  assumption,  cost  functional  stabilizability  and  stabilizability  are 
equivalent.  In  general,  uniform  stabilizability  and  uniform  detectability  are  required 
for  the  convergence  of  rUi00  to  IIoo  as  h  tends  toward  zero. 

THEOREM  3.4.  Let  Assumption  (B)  hold.  Suppose  further  that  Conditions 
(Al)-(A4)  hold  on  any  bounded  subinterval  of  [0,oo).  If  the  continuous  time  system 
and  the  sampled  time  systems  are  uniformly  stabilizable  and  uniformly  detectable, 
then,  the  unique  solution  of  the  infinite  horizon  Riccati  difference  equation  (3.4) 
converges  to  the  solution  IIoo  of  the  infinite  horizon  Riccati  integral  equation  (3.3)  as 
h  tends  toward  zero.  The  convergence  is  uniform  in  time  on  any  bounded  subinterval 
of  1 0,  oo). 


Proof.  By  Theorem  3.3,  uniform  stabilizability  and  uniform  detectability  imply 
exponential  stability  of  the  optimal  feedback  systems  (i.e.,  Definition  3.2),  uniformly 
over  all  sampled  systems  with  0  <  h  <  ho.  Therefore,  by  Theorem  3.2,  we  obtain  the 
desired  convergence.  □ 

Most  control  systems  of  interest  in  engineering  practice  are  stabilizable  and  de¬ 
tectable.  In  fact,  in  modeling  many  control  systems  of  practical  interest,  a  realis¬ 
tic  description  of  the  physical  system  frequently  necessitates  stabilizability  and  de¬ 
tectability  of  the  system  model  (see,  for  example,  [BKS],  [BKSW]).  Investigation  of 
stabilizability  and  detectability  of  particular  classes  of  evolution  systems  has  gener¬ 
ated  several  interesting  mathematical  problems  (see,  for  example,  [C],  [ L j ) .  However, 
in  the  context  of  approximation,  we  usually  assume  that  the  original  control  system 
is  stabilizable  and  detectable.  An  important  issue  here  is  whether  or  not  a  given 
time  discretization  algorithm  is  capable  of  preserving,  uniformly,  these  properties, 
and  therefore  provide  discrete  time  convergent  approximations  for  the  optimal  feed¬ 
back  operators.  In  the  remainder  of  this  section,  we  attempt  to  address  this  issue  for 
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some  particular  discretization  algorithms  and  indicate  some  other  possible  approaches 
to  this  problem. 

Assume  that  the  control  system  defined  in  Equation  (2.1)  is  stabilizable  and  de¬ 
tectable  with  respect  to  the  cost  functional  (3.1).  Thus,  there  exist  bounded  operator- 
valued  functions  K(-)  :  [f0»co)  ^  L[H;U )  and  V(-)  :  [to,oo)  ^  £(#)  such  that  the 
evolution  systems  TV,  TV,  corresponding  to  the  perturbations  of  T  by  BK  and  V Q1^2, 
respectively,  are  exponentially  stable.  That  is,  there  exist  constants  M,a  >  0  such 
that  ||7V(t,«)IU(/r)  <  Mexp{-a(*  -  s)}  and  !|7V(M)||l(J7)  <  M  exp{-a(t  -  s)},  for 
all  0  <  .s  <  2  <  oo.  By  definition,  the  evolution  operators  TV,  and  TV  satisfy 

(3.7)  TK(t,s)<f>  =  T[t,s)<t>  + 1,  T{t,r))B{r))K{r))TK{r,,s)<t>di), 

(3.8)  Tv{t%s)4>  =  T[t,s)4>  +  J*  T{t,V)V{T})Ql/2{T))Tv{v,s)<i>drj, 


for  all  <{>  €  H  and  for  all  0  <  s  <  t  <  oo.  Consider  the  zero-order  hold  discretization 
described  in  the  Section  2.  For  each  k  >  0,  the  operators  Ah(k),Bh[k)  are  defined  by 


(3.9)  Ah(k)  =  T{{k  +  l)h,kh), 

l  f(k+l)h 

(3.10)  Bh(k)  =  -/  T((k+l)h,r,)B(n)dr}, 

h,  Jkh 


with  the  discrete  evolution  systems  T*  *,  Ty ^  then  given  by 


(3.11) 

Tx.hihj)  = 

\YYk~JMHW  +  hBh(k)K(kh)},  i>j , 

(3.12) 

Ty,h{i,j)  =  | 

/,  *  =  J, 

YYkJMhik)  +  hV(kh)Qh(k)'/2},  i  >  j. 

If  the  discrete  time  evolution  systems  TV,/t»TV,/»  are  uniformly  exponentially  stable  for 
all  0  <  h  <  h0  for  some  h0  >  0,  then,  these  sampled  time  systems  are  uniformly 
stabilizable  and  uniformly  detectable.  Using  (3.7)  and  (3.8),  the  evolution  systems 
TV,  and  Ty  satisfy 

TK(ih,jh)  =  n 

k=j 


r{k+l)h 

T((k  +  l)h,kh)+  T((k  +  l)h,  rf)B(r})K(r})Tfc(r?,  kh)drj  , 

J  kh 


and 

Tv(ih,jh)  =  n 

k  =  ] 


r(k+l)h 

T((k  +  l)h,kh)  +  /  T{{k  +  l)h,rj)V (tj)Q(ti)iI2Tv(ti,  kh)dr] 
Jkh 


for  0  <  j  <  i  <  oo.  Therefore,  and  Ty ^  can  be  considered  as  perturbations  of 

TV  and  TV,  respectively.  In  fact,  we  have 


(3.13) 

TkA'J) 

(3.14) 

Ty,h{i,j ) 

l[{TK{{k  +  l)h,kh)  +  h*h(k)}, 


k=j 


i  — 1 


H{Ty((k  +  l)h,kh)  +  h*h(k)}, 
k~] 
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for  0  <  j  <  i  <  oo,  where 

$h(k)  =  |  (C+l)hT^k  +  l)h,n)B{rt)[K{kh)  -  K{Ti)Tv{n,kh)]dvj  , 

and 

¥*(*)  =  ^V(*fc)QJk(fc)1/1  -  \  j^+l)h  T((k  +  l)h,T,)V(r,)Q(r,)l/2Tv{r,,kh)dr^  , 

for  k  >  0.  Let  0  <  u  <  a  and  define  Tjc,h{i,j)  —  exp{w(t  —  y)h)  }!*>(*, j),  0  <  j  < 
t  <  oo  and  Tx(t,.s)  =  exp{u>(t  —  5)}T/f(t,s),  0  <  5  <  t  <  oo.  We  define  Tv  and  TV,/i 
analogously.  It  is  not  difficult  to  verify  that 

llfje(tts)llz(B)  <  M,  ||Tv(f,5)||x,(H)  <  M. 

Multiplying  both  sides  of  (3.13)  and  (3.14)  by  exp{w(t  —  j)h)},  and  rewriting  these 
equations  in  a  variation  of  constants  form,  we  obtain 

TxAiJ)  =  fK(ih,jh)  +  h''£fK(ih,(ki  1)h)euk*h(k)fK,h(kJ), 

k=j 
i- 1 

fv,h(  i,j)  =  fv{ih,jh)  +  h'£>fv{ih,(k  +  l)h)e“h*h{k)fv,h{k,j). 

k=j 

If  there  exists  a  constant  ho  >  0  such  that  for  all  h  <  ho,  exp{a;/i}||$h(fc)  ||l(H)  <•  w/2Af 
and  exp{w/i}||'I'/l(A:)||£,(tf)  <  w/2 M,  then, 

\\fK,h(i,j)\\L(H)  <  M  +  h^M  ■  ^\\TK,h(k,j)h{Hb 

\\Tv,h{i,j)\\L(H)  <  M +  h'^M  ■ —\\fv,h[kJ)\\L{H)- 

The  discrete  Gronwall  inequality  then  yields 

l|rv,»(»',y)||i(H)  <  . 

Therefore,  Tk ,h,  and  Tv,h  are  uniformly  exponentially  stable  for  all  0  <  h  <  ho* 

It  is  not  difficult  to  see  that  for  each  k  >  0,  $>,(&)  and  'J'h(k)  converge  strongly 
to  zero  as  h  tends  toward  zero.  We  can  obtain  convergence  in  norm  if  the  rank  of  the 
operator  valued  functions  $/,(k)  and  'i'h(k)  is  finite. 

DEFINITION  3.5.  (Finite  rank  operator-valued  function)  Let  X  and  Y  be  Hilbert 
spaces  with  inner  products  <  >x  and  <  •,•  >y,  respectively.  An  operator-valued 
function  PF(-)  :  [0,  oo)  »-»  L(X,  Y)  is  said  to  be  continuous  and  to  have  finite  rank,  if 
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there  exist  continuous  vector-valued  functions  /*(•)  :  [0,  oo)  ►  X  and  gk(-)  :  [0,  oo) 

Y,  k  =  1,  •  ■  • ,  n  with  n  <  oo,  such  that  for  all  i£X, 

w(t)x  =  J2<  fk{t),x  >Y  gk{t). 

k=l- 

We  define  the  following  condition. 

(F)  (Finite  Rank  Stabilizability  and  Detectability  Condition)  There  exist  finite  rank 
continuous  operator-valued  functions  K(-),  V(-)  such  that  the  perturbed  evo¬ 
lution  systems  Tk  and  Ty  are  exponentially  stable. 

LEMMA  3.4.  Suppose  that  Conditions  (A1)-(A4)  hold.  If  the  finite  rank  con¬ 
dition  (F)  is  satisfied,  then  on  any  finite  subinterval  of  [0,  oo),  the  operator-valued 
functions  and  constructed  from  and  'I'/,  in  the  usual  manner,  converge 
uniformly  to  zero  in  the  uniform  operator  norm  as  h  tends  toward  zero. 

Proof.  We  consider  ^  only,  the  argument  for  is  analogous.  Using  the  finite 
rank  condition,  we  write 


V {*)<!>  =  Yi  <  ffk(t), 

k=l 

with  ft  and  gk  continuous  for  k  =  1,  •  •  •  ,n.  It  follows  that 

V(ih)Qh(i)ll2<t>  <  Qh{i)1/2 fk{ih),<t>  >h  9k{ih ), 

*= l 

for  i  >  0,  and 


T{t,r))V(r])Q(T]y/2Tv{r),s)<t)  =  <  Tv{v,s)'Q{Tl)1/2fk{v),<l>  >H  T{t,rj)gk[rj), 

k=  1 

for  0  <  s  <  r]  <  t  <  oo.  Therefore,  we  have 

*k{t)4>  =  V({t/h}h)Qh(ty/2<f> 

— r  f  T{([t/h]  +  l)h,T])V(i])Q{rj)l/2Tv(rj,[t/h}k)<pdr) 

h  V [t/HjA 

1  "  r([t/h\+l)h  , 

=  r£/,,  {<  Qh{t) 1  »»([«/*]*) 

-  <  Tv(r],[t/h]hYQ(r))1/2fk(r}),<t>  >H  T{{[t/h\  -f  l)h,  rj)gk(fj) }  dr). 

By  adding  and  subtracting  the  term  <  Tv{n,  [t/h]h)*Q(r?)1/2/Jfc(r;)1  <£  >H  gk[[t/h]h) 
under  each  of  the  above  integral  signs,  and  using  the  Schwartz  inequality,  we  obtain 
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the  following  estimate 


1  A  (■{{t/h\+i)h 

<  tM,,  {u>k{t,ri)\\g{[t/h}h)\\H 

h  J[t/h\h 

+vh{t,ri)\\Tv{r},[tlh\hyQ{r)Yl2fk{ji)\\H]  \\4>\\Hdrj 
1  A  /•([</*]+!)* 


where 


Wh{t,ri)  =  \\Qh{t)1/2fki\t/h]h) -Tv{r},[t/h}hyQ{r})l/2fk(rj)\\II, 

Vh{t,ri)  =  \\g{\tlh\h) -T{{[tlh\  +  l)h,r})g{r})\\H,  and 

uh(t,rj)  =  wh(t,ri)\\g{{tlh\h)\\H  +  vk(t,i))\\Tv{v,[t/h}hyQ[r))1/2fk(ri)\\H. 


Since  the  functions  fk  and  gk  are  continuous  on  any  bounded  subinterval  [a,  6]  of  [0,  oo), 
and  for  any  e  >  0,  there  exists  6  >  0  such  that  ||A(t)  —  /ic(s)\\h  <  e  and  ||j7it(<)  — 

<  6  for  all  t,  s  €  [a,  6]  with  jt-s|  <  6  and  k  =  1,  •  •  • ,  n.  Then,  the  boundedness 
of  the  operator  families  T,Tv,V,Qk,Q  and  the  uniform  strong  convergence  of  Qh  to 
Q  implies  that  for  any  bounded  subinterval  [a, 6]  of  [0,  oo),  and  for  any  given  constant 
€  >  0,  we  can  find  h0  >  0  such  that  for  all  0  <  h  <  hQ  and  t  6  [a,  b],  the  functions 

<  e  for  77  €  [t,t  +  h\  and  t  €  [a,6j.  Consequently  ||4rh(t)||i(H)  <  e  for  all 

f€[a,6j.  □ 

We  can  extend  the  uniform  convergence  on  finite  time  intervals  to  uniform  conver¬ 
gence  on  the  infinite  time  interval  by  assuming  certain  periodicity  (in  particular  time 
invariance)  of  the  evolution  system  T  and  the  operator-valued  functions  B,Q,Qh,  K 
and  V.  In  fact,  the  periodicity  assumption  implies  that  are  also  periodic 

functions  of  time. 

THEOREM  3.5.  Assume  that  the  evolution  system  T  and  the  operator-valued 
functions  B,Q,R  are  strongly  continuous  and  periodic  with  the  same  period,  9.  Sup¬ 
pose  further  that  the  periodicity  of  Q  is  preserved  by  Qk  for  the  sampled  time  systems. 
If  the  finite  rank  condition  (F)  holds  for  some  9-periodic  functions  K  and  V ,  then 
the  discretization  defined  in  (3.9)  and  (3.10)  generates  uniformly  stabilizable  and  uni¬ 
formly  detectable  sampled  control  systems  for  sampling  periods  h  with  0  <  h  <  h0  for 
some  constant  h0  >  0. 

The  periodicity  assumption  is  trivially  satisfied  in  a  large  number  of  practical  ex¬ 
amples,  in  particular,  it  is  satisfied  for  all  time  invariant  systems.  However,  the  finite 
rank  assumption  says,  in  essence,  that  only  a  finite  number  of  modes  of  the  state 
vector  are  unstable  in  the  absence  of  control.  Indeed,  in  the  case  of  evolution  systems 
corresponding  to  a  hyperbolic  partial  differential  equation,  there  exists  examples  in 
which  if  the  finite  rank  condition  is  not  satisfied,  all  sampled  systems  are  not  stabi¬ 
lizable  even  though  the  continuous  time  control  system  is  stabilizable.  For  parabolic, 
compact,  evolution  systems,  the  spectral  properties  of  the  evolution  system  provide 
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valuable  additional  structure.  In  this  case,  an  approach  which  does  not  require  the 
finite  rank  condition,  similar  to  the  one  used  in  [Rj,  can  be  applied.  The  results  using 
this  type  of  argument  will  be  reported  on  elsewhere.  However,  even  in  the  case  of 
parabolic,  compact,  evolution  systems,  since  the  unstable  spectrum  consists  of  only 
a  finite  number  of  isolated  points,  it  would  be  interesting  to  know  whether  these  sys¬ 
tems  can  be  stabilized  via  finite  rank  feedback.  If  the  answer  is  affirmative,  then  the 
arguments  presented  here  may  not  be  as  restrictive  as  they  seem.  For  other  discretiza¬ 
tion  schemes,  the  uniform  stabilizability  and  uniform  detectability  of  the  generated 
sampled  systems  remains,  in  most  cases,  an  open  question. 

4.  Examples  and  Numerical  Results.  In  this  section  we  present  and  briefly 
discuss  some  of  our  numerical  findings  which  serve  to  illustrate  our  convergence  results 
in  the  context  of  a  variety  of  distributed  parameter  control  systems.  In  particular, 
we  consider  the  infinite  horizon  optimal  control  or  regulation  of  a  heat  or  diffusion 
equation,  a  delay  or  hereditary  system,  and  a  flexible  structure  in  the  form  of  a 
cantilevered  Voigt-Kelvin  viscoelastic  beam  with  tip  mass. 

In  all  of  the  examples  to  follow,  we  consider  time  invariant  systems  only,  and 
obtain  the  discrete  or  sampled  time  operators  from  the  corresponding  continuous 
time  operators  via  Th  —  T(h),  Bh  =  h~l  T{t)Bdt,  Qh  =  Q,  and  Rh  =  R ,  for 
h  >  0  (i.e.,  via  zero-order  hold  sampling).  In  order  to  solve  the  resulting  infinite 
dimensional  continuous  and  discrete  time  LQR  problems,  we  introduced  some  form 
of  state  discretization  (i.e.  either  modal  or  spline  based  Ritz-Galerkin  techniques) 
which  were  known  to  yield  convergence  in  the  closed-loop  problem.  By  choosing  the 
state  discretization  sufficiently  fine,  we  could  assume  that  we  obtained  a  reasonably 
accurate  finite  dimensional  approximation  to  the  solution  of  the  infinite  dimensional 
LQR  problems. 

The  resulting  finite  dimensional  continuous  and  discrete  time  LQR  problems 
(more  precisely,  the  matrix  algebraic  Riccati  equations)  were  solved  using  either  eigen¬ 
vector  (in  the  continuous  time  case,  also  known  as  Potter’s  method,  see  [KS])  or  Schur 
vector  (for  the  discrete  time  problems,  see  [PLS])  decomposition  of  the  Hamiltonian 
matrix.  All  computations  for  the  first  two  examples  were  carried  out  on  an  IBM  PC 
AT.  The  flexible  structure  problem  was  solved  on  an  IBM3090,  although  it  too  could 
have  been  solved  on  a  personal  computer. 

In  each  of  the  examples  below,  the  control  systems  are  time  invariant  and  the 
control  space  U  is  finite  dimensional.  In  fact,  U  =  R.  Thus,  the  optimal  feedback 
gains,  F  and  Fh,  are  elements  in  L(H,  R).  That  is,  they  are  bounded  linear  functionals 
on  H.  Consequently,  they  admit  representors,  respectively  /  and  /*,  in  H  with  F<p  =< 
f,<p  >ff  and  Fh<p  =<  >H,  for  <p  €  H.  The  elements  /  and  fh  in  H  are  referred 

to  as  the  optimal  continuous  or  discrete  time  functional  feedback  control  gains.  The 
finite  dimensionality  of  the  control  space  U  also  implies  the  uniform  stabilizability 
of  the  sampled  systems  when  the  continuous  time  systems  are  stabilizable  (recal1 
Theorem  3.5).  Our  convergence  result  implies  that  lim>,_o+  Fk<P  —  F<p  for  <p  €  H. 
Note  that  when  U  is  finite  dimensional,  this  is  equivalent  to  lim(v_0+  Fh  =  F  in 
the  uniform  norm  topology  on  L(H,U)  and  limh_0+  fh  =  / in  H.  It  is  this  latter 
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convergence  of  the  functional  gains  which  we  shall  exhibit  in  our  plots  below. 
Example  4.1.  We  consider  the  scalar  or  one  dimensional  heat  or  diffusion  control 
system 

d  d2 

—  x(t,r7)  =  a—x(t,r/)  +  6x[cii£a|(r?)u(f),  0  <  rj  <  l,t  >  0, 

with  the  Dirichlet  boundary  conditions 

x(t,0)=x(t,  1)  =  0,  t  >  0, 

at  r]  =  0  and  t]  =  1  where  a  >  0,  b  G  R,  0  <  £j  <  £2  <  1,  and  x»  denotes  the 
characteristic  function  on  the  set  S.  We  take  the  performance  index  to  be 

J(u)  =  j  {J  qx(t,rj)2dri  +  ru(t)2}dt 

with  q  >  0  and  r  >  0. 

In  this  case  we  have  H  =  L2( 0, 1),  U  =  R,  A  :  Dom(.<4)  C  H  >—>  H  given  by 

Acp  =  aD2ip  for  cp  €  Dom(yl)  =  H2(0, 1)  Q  Hq(0,  1), 

B  e  L(R,  H)  given  by  ( Bv){tj )  =  bx[(uti](ri)v,  0  <  rj  <  1,  v  €  R,  Q  E  L(H)  given 
by  Q  —  ql,  and  R  €  L(U )  given  by  R  =  rl ,  where  /  denotes  the  identity  map  on 
R.  We  note  that  (T(t)  :  t  >  0),  the  semigroup  of  bounded  linear  operators  on  H 
with  infinitesimal  generator  A,  is  parabolic  and  uniformly  exponentially  stable.  Thus 
the  continuous  time  pairs,  {A,B}  and  {Q,A}  are  trivially  stabilizable  and  detectable 
and  the  discrete  time  pairs,  {T/,,5/,}  and  { Qh ,1\}  are  uniformly  stabilizable  and 
detectable  as  well. 

Setting  a  =  0.1,  6  =  1.0,  q  =  1.0,  r  =  1.0,  e\  =  0.21,  and  e2  =  0.275,  we 
obtained  the  plot  of  the  functional  gains  /  and  fh  in  L2(0, 1),  for  various  values  of 
h  >  0,  given  in  Figures  4.1  and  4.2.  Those  in  Figure  4.1  were  obtained  via  a  modal 
(i.e.  sin(fc7rx), k  =  1,2, •••,1V)  state  discretization  with  N  =  20  modal  elements. 
For  the  gains  in  Figure  4.2,  we  used  linear  B-spline  elements  (i.e.  “hat”  functions) 
defined  with  respect  to  a  uniform  partition  of  [0,  l]  into  N  =  20  subintervals  of  equal 
length.  Convergence  of  these  state  approximations  and  the  corresponding  closed-loop 
solutions  to  the  control  problem  is  well  known  (see,  for  example,  [G],  [GRj). 
Example  4.2.  In  this  example,  we  consider  the  scalar,  single  input  hereditary  control 
system 

(4.1)  i(t)  =  oo  x(f)  +  dix(f  —  1)  +  bu(t) 

where  a0,al,b  G  R.  We  take  the  performance  index  to  be 

J(u)  =  [  {qx2[t)  +  ru2(t)}dt 
Jo 

with  q  >  0  and  r  >  0. 
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Table  l 

Head  gains  for  hereditary  system. 


Sampling  period  h 

■igT.FTWM 

nr1 

3.76185 

1(T2 

4.35007 

1CT3 

4.41577 

10’4 

4.42241 

1(T5 

4.42308 

nr6 

4.42314 

Continuous  time 

4.42315 

The  abstract  Hilbert  space  formulation  for  linear  hereditary  control  systems  is 
well  known  (see,  for  example,  [BB]) .  We  let  H  =  R  x  L2(— 1,0),  17  =  R  and  set 
A  :  Dom(A)  C  H  H  to  be  A(rj,<p)  =  ( OqT}  +  ai<p{— 1), D<p)  for  {rj,<p)  6  Dom(A)  = 
{(£,  tp)  €  H  :  tp  e  Hl(— 1,0),  £  =  0(0)}.  The  operator  A  is  the  infinitesimal  generator 
of  the  Co-semigroup  of  bounded  linear  operators  on  H,  (T(t)  :  t  >  0},  given  by 
T(t)(r],ip)  =  (x(f),xt)  where  x  is  the  solution  to  (4.1)  with  u  =  0  and  corresponding 
to  the  initial  data  x(0)  =  rj,  x(9)  =  < p(9),  —  1  <  8  <  0,  and  xt  €  T2(— 1,0)  is  the 
past  history  of  x  from  t  back  to  t  —  1.  That  is  xt(0)  =  x(t  +  *).-!<  8  <  0.  We  let 
B  €  L(R,H)  Q  e  L(H),  and  R  €  L(U)  be  given  by  Bv  =  (6v,0),  Q(rj,<p)  =  (qr],  0), 
and  Rv  =  rv ,  respectively. 

To  solve  both  the  continuous  and  discrete  time  LQR  problems  we  employed  a 
piecewise  constant/linear  spline  hybrid  finite  element  scheme  developed  by  Ito  and 
Kappel  in  [IK].  Setting  a0  =  a1  =  6  =  g  =  r  =  l,  and  with  a  state  discretization 
level  in  the  Ito-Kappel  scheme  taken  to  be  N  =  20,  we  obtained  the  R  x  L2(— 1,0) 
functional  gains,  /  =  (Z0,/1)  and  / h  =  (/°,//|)  for  various  values  of  h  >  0,  tabulated 
and  plotted  in  Table  1  and  Figure  4.3  below.  We  note  that  for  this  choice  of  the 
parameters  a0  and  ai,  the  open  loop  system  has  an  eigenvalue  with  positive  real  part. 
Consequently  the  system  (4.1)  is  open-loop  unstable.  It  is  not  difficult  to  argue  that 
the  pairs  {A,B}  and  {Q,  A}  are  respectively  stabilizable  and  detectable.  Also,  since 
the  operators  B  and  Q  are  of  finite  rank,  there  exists  hQ  >  0  such  that  for  all  sampling 
periods  h  <  h0,  the  sampled  control  systems  are  uniformly  stabilizable  and  detectable 
in  h. 

Example  4.3.  We  consider  the  control  of  the  small  amplitude  transverse  vibration  of 
a  cantilevered  Voigt-Kelvin  viscoelastic  beam  with  tip-mass.  The  relevant  dynamics 
are  described  by  the  hybrid  system  of  ordinary  and  partial  differential  equations 

piax(t,T1^  +  cId^dix^t,r,')  +  EIJ^x^'ri)>  =0,T7  €  t0,1)’ 

~  cId^dix{t' “  EI^x{t' 1}  =  bu^' 
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FlG.  4.3.  Functional  gains  for  delay  cquaiion.  (a)  h=0.1,  (b)  h=0. 01,  (c)  k— 0.001,  (d)  h— 0.0001, 
(e)  Continuous  time. 


for  t  >  0,  the  essential  (or  stable)  boundary  conditions  at  q  =  0 

Q 

xlt,  0)  =  0,  —x(t,  0)  =  0,  t  >  0, 

dr] 

and  the  natural  (or  unstable)  boundary  condition  at  r?  =  1, 

~  EI J^x(,’1)  =  °'  ,>0- 

In  the  above  equations  p  >  0  is  the  linear  mass  density  of  the  beam,  /  >  0  is  the 
beam’s  cross  sectional  moment  of  inertia,  c  >  0  is  the  viscosity  coefficient,  E  >  0  is 
Young’s  modulus,  m  >  0  is  the  mass  of  the  tip  mass,  and  b  £  R  is  a  constant. 

We  take  an  energy  based  performance  index: 

J{u)  =  C{lEIC{h<t',,))1,h,+\m(Ttxlt'1))1 

+  L  +  ru^2 } dt- 

Once  again  the  abstract  Hilbert  space  formulation  of  this  problem  is  standard.  We 
let  H  =  Hl(0, 1)  x  R  x  Z,2( 0, 1)  where  H\{ 0,  l)  =  {<p  G  H2( 0, 1)  :  <p(0)  =  D<p{ 0)  =  0}, 
and  endow  H  with  the  energy  inner  product 

<  (<Pl,J7l,tS),  0^2,  >H  =  El  [  D2<p xD2ip2 

Jo 

-trnriir)!  +  p 

Jo 

The  operator  A  :  Dom(A)  C  H  H  is  given  by  =  (t/>,  c/D3t/>(l)  + 

EID3<p(l),-cID4xlj  —  EID4<p)  for  {<p,r],xl))  6  Dom(A)  =  €  H  :  rj)  G 

Hl(Q,  1),  77  =  ^(l),cJDV+£/DVeff!(0,l),c/D^(l)  +  £/i)V(l)  =  0}.  We  take 
t/  =  R  and  define  B  e  L(R,  H)  by  Bv  =  (0,6v,0).  We  let  Q  €  L[H)  and  R  €  L{U) 
be  given  by  Q  =  (1/2)/#  and  12  =  r/(/,  where  /#,  and  /y  denote,  respectively,  the 
identity  operators  on  H  and  U. 

It  can  be  shown  (see  [GA])  that  A  is  the  infinitesimal  generator  of  a  uniformly  ex¬ 
ponentially  stable  analytic  semigroup.  Thus  once  again  stabilizability  and  detectabil¬ 
ity  for  the  continuous  time  problems  trivially  follows  as  does  the  uniform  stabilizability 
and  detectability  for  the  discrete  time  problems. 

We  employed  a  standard  cubic  spline  based  Ritz-Galerkin  finite  element  scheme  to 
approximate  or  finite  dimensionalize  the  continuous  and  discrete  time  LQR  problems 
(see  [GAJ,  [GR]) .  Setting  p  =  0.1,  El  =  1.3333  x  1(T4,  cl  =  1.3333  x  1(T7,  m  =  1,  b  = 
1,9  =  1,  and  r  =  1  and  with  N  =  9  cubic  spline  elements,  we  obtained  the  functional 
gains  /  =  (/°,  /*,  /2),  fh  =  (/£,  /y|,  f£)  g  H  exhibited  in  Table  2  and  Figure  4.4  below. 
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Table  2 

Tip  gams  for  beam  equation. 


Sampling  period  h 

Tip  Gain,  /' 

1.000 

0.12181 

0.500 

0.12003 

0.010 

0.11798 

0.005 

0.11796 

0.001 

0.11794 

Continuous  time 

0.11793 

FlG.  4.4.  Functional  gams  Jor  beam  equation,  (a)  Displacement,  we  plot  D~  /"  to  exhibit  the  H 2 
convergence;  ( b )  Velocity. 


5.  Summary  and  Concluding  Remarks.  We  have  investigated  and  estab¬ 
lished  the  convergence  of  solutions  to  discrete  or  sampled  time  linear  quadratic  regu¬ 
lator  problems  and  the  associated  Riccati  equations  for  infinite  dimensional  systems 
to  the  solutions  to  the  corresponding  continuous  time  problem  and  associated  Riccati 
equation,  as  the  length  of  the  sampling  interval  tends  toward  zero.  We  have  con¬ 
sidered  both  the  finite  and  infinite  time  horizon  problems  and  carried  out  numerical 
studies  involving  a  variety  of  distributed  parameter  control  systems  in  order  to  ob¬ 
serve  how  well  our  theoretical  results  predict  what  actually  takes  place  in  practice.  In 
the  context  of  the  finite  time  horizon  problem,  the  assumption  of  strong  continuity  on 
the  operators  which  define  the  control  system  and  performance  index,  together  with  a 
stability  and  consistency  hypothesis  on  the  sampling  scheme,  are  sufficient  to  establish 
the  strong  convergence  of  the  Riccati  operators,  feedback  gains,  optimal  control  laws, 
and  optimal  trajectories,  with  some  degree  of  uniformity  in  time  over  the  compact 
interval  of  interest.  For  the  infinite  time  horizon  problem,  we  require  the  additional 
assumption  of  stabilizability  and  detectability,  uniformly  with  respect  to  the  length 
of  the  sampling  interval.  We  have  shown  that  this  condition  can  be  verified  when 
zero-order  hold  sampling  is  employed  and  the  continuous  time  system  is  stabilizable 
and  detectable  by  finite  rank  feedback.  We  also  have  a  result  for  parabolic  systems, 
but  this  will  be  reported  on  elsewhere. 

Several  interesting  open  questions  related  to  the  results  we  have  presented  here 
remain  open.  For  example  the  inter-relation  between  stabilizability /detectability  for 
the  continuous  and  sampled  time  systems  in  a  more  general  setting  and  under  more 
general  sampling  schemes  (A-Stable  Pade,  for  example)  requires  further  study.  Also, 
convergence  under  simultaneous  and  independent  state  (space)  discretization  (i.e. 
finite  difference  or  finite  element  approximation)  and  temporal  sampling  should  be 
investigated.  It  would  not  be  difficult  to  extend  the  results  we  presented  here  to 
handle  certain  “coupled”  state  and  time  discretizations.  Finally,  a  study  similar  to 
the  present  one  could  be  carried  out  for  the  LQG  estimator  and  compensator  problems. 
We  have  not  as  of  yet  looked  at  these  problems,  but  suspect  that  similar  results  to 
those  given  above  could  be  obtained. 
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Appendix  A. 

(i)  Continuity  of  Ph:  Let  <j>  6  U.  Then  since  Ph<j>  is  constant  on  each  of  the 
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intervals  =  [jh,  ( j  +  l)/i),  we  have 


IIJWIIL  = 


< 


JO 

E  fcll h~l  f  Hs)ds\\2 

J=0  JI> 

E  b'1  jr  M')\\*ds  f  ds 

j=0 

/"'*  n*wir*  <  mi 

Jo 


(ii)  Convergence  of  Pk'-  First,  let  us  consider  a  continuous  function  4>  €  U.  Then,  for 
all  t  £  there  exists  /i0(t)  >  0  such  that  for  all  h  <  h0(t),  we  have  t  £  [0,f/A]. 

That  is,  for  all  t  £  [0, t /], 


(Ph<t>)(t)  =  h  1  Ji<f>(s)ds 

for  h  small  enough,  where  /  is  an  interval  of  length  h  containing  t.  The  Bochner 
integral  is  equal  to  the  Rieman  integral  for  continuous  functions,  and  merefore,  for 
all  t  £  [0,£/),  it  follows  that 

^lim  (P^)(t)  =  <£(*). 

Now  consider  an  extrapolation  operator,  Eh  :  Uh  >-*  U,  defined  for  all  u  £  Uh 
as:  Ehu{t)  =  u(t),  for  t  £  [0, tfih],  Ehu(t)  =  0  for  t  £  It  is  evident  that 

( EhPh4>)(t )  converges  to  <f>(t)  as  h  tends  to  0  for  all  t  £  [0, tfh).  Since 


[*'  \\(EhPh4>)(s)\\2ds  =  ||P^||Btf 
Jo 


from  (i),  we  conclude,  via  the  dominated  convergence  theorem,  that 


Hm  \\Ph<l>\\uk  =  H^llu- 

fl — *0’’ 

Then  from  the  uniform  boundedness  of  Ph  and  the  density  of  the  continuous  functions 
in  U,  we  obtain 


lim  \\P^\\Uh  =  ||4||u 

A— 

holds  for  all  <f>  £  U. 

(iii)  For  all  u  £  Uh  we  have  Eh u  £  U ,  PhEhu  =  u,  and  \\PhEhu\\uK  =  ||^u||u. 


Appendix  B. 

We  shall  show  that 


lim  || ~  PhB,<t>\\uk  =  0, 


uniformly  in  s  for  s  6  [0,f/],  and  4>  Q  H.  Analogous  arguments  can  be  used  to  show 
that 


lim  ||  Hh,.PhU  -  PhZ,u\\u.  =  0 

h— 0+ 


and  that 


£?+  ll®MP*tt  ~  b;uII»  =  0 

uniformly  in  s  for  s  Q  [0,i/],  and  each  uQ.U. 

From  the  definitions  of  8,  and  8,  given  in  (2.2)  and  (2.11)  respectively,  it  is  easy 
to  observe  the  following  for  any  <f>  €  H. 

(i)  ||(Bi0)(OI|t;  is  continuous  with  respect  to  s  for  all  t  >  s.  Therefore,  from  the 
uniform  boundedness  of  ||(B*0)(t)||i/  for  all  0  <  s  <  t  <  tf,  we  conclude  that 
8,<j>  is  a  continuous  function  of  s  in  the  U  norm. 

(ii)  By  condition  (A1)-(A4),  it  is  easy  to  see  that 

*(M)  =  l|B.,**(t)-M(*)lla 

converges  to  zero,  uniformly  in  A(2,t/).  By  the  unuorm  boundedness  of 
||Bh,4<?H*)||c/  for  all  h,  we  conclude  that 

\\B,,h4> -  6ml  =  /'  + ['*  ii 6.j,m  -  B.mihdt 

J[s(k\h  J\s/h]h 

converges  to  zero,  uniformly  in  s. 

Since  we  have 

II Bh>  -  PhBMW  <  II ~  Mk  +  || (A  -  /)Mk> 

the  strong  continuity  of  8,,  and  the  strong  convergence  of  Ph  to  the  identity 
on  U,  yield  the  desired  uniform  convergence  result. 
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